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PREFACE 

The Office of Student Financial Assistance (OSFA) of the Depart ent of 
Education has contracted with Advanced Technology, Inc, of McLean Virglnia,-«nd 
its subcontractor, Wcstat, Inc^ of Kockvilie, Maryiand, to conduct a three-year 
quaUty control project (Contract No-j 300-S(M)952). The focus of the project i$ the 
PeM Grant Program, the Urgest of the student grant programs administered by 
OSFA. The objective of Stage Two (Part One) of the project is to design a quaUty 
control system to measure and analyze program perform/ince. The reports 
completed to date under Stage Two (Part One) includes 
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EXECUTIVE SUMMARY 

* During the past two yman the Qtlioe of Student Fir^ancial Assistance has peen 
engaged in an effort to deveioo an internal qua lity improvement program. The 
purpose of the program Is to develop a s ystemaiitf "teroach to i dentifying , 
measurjng, and correcting errors or tendencies toward errors in the student aid 
delivery system. The design for this program includes both a measurement 
com p on e nt and a corrective action component. This r^>ort de8crll>es the purpose 
and direction of the OSFA Cj^iality improvement program. The report has four 
chapters. 

Chapter 1 provides background on the quality improv«nent program. Some of 
the basic points covered indudet 

o A review of the literature suggests quality control and corrective action 
can be most effective when targeted at the most error-prone areas. 

o The overall OSFA quality improvement program has QC measurement 
and corrective action components, both deigned to target technical 
assistance at high error-prone areas* 

Chapter 2 reviews the general approach to quality improvement developed 
during the Stage n Pell Cgmnt Quality Control study. This approach consists of the 
following key elements: 

o An overall strategic approach that enables OSFA to target technical 
assistance on high error-prone areas, while proceeding with an overall 
design strategy. 

o An em^^iasis on both manual and automated quality control measure- 
ment, enhancing existing data bases and procedures to the extent 
possible. 

o An emphasis during the first year of the quality improvement program on 
the development of the measurement component of the program. 



Chapter 3 presents the overaJl design for the OSFA quality improvement 
program. First the opportunities lor quality improvement in the current delivery 
system. Marginal improvements to critical activities in the current system can 
alleviate some of the most seriously negative effects of the current syHam, 
including fund control and availahiUty of program information. 

Second an overall strategy for the OSFA quality improvement program is 
presented. OSFA is undertaldng a quality improvement program that concentrates 
on: 

o Institutional Quality Contrcd, which is encouraged by Federal regulations 
and facilitated by institutional Quality Control guidelines. 

o External Quality Control, whidi can provide OSFA with an ongoing 
measurement of overall error rates for all progrMis. 

o Internal OSFA Quality Control, which includes a supplementative effort 
to identify error prone activitiies, develop measurement mechanisms for 
these activities, and identify corrective action options. 

o A Qualhy Control Management Information Systems (QCMIS), that will 
provide an overall OC reporting and information system 

Third, the critical targets for internal quality improvement are identified and 
measures are proposed. The critical activities, some of which were addressed during 
Stage 11 of the Pell QC Study, provide OSFA with a road map or future internal 
quality improvement. 

Fourth, the next steps in the OSFA quality knprovement process are 
considered. These includes 

o selection of new targets for technical assistance from the list of initial 
activities. 

o development of a systematic approach to quality improvement for 
critical activities, which Is ad^wed in the final chapter. 

o development of ihe CMIS framework which would provide OSFA with a 
well-defined QC information and reporting system. 
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CNpter # piwntf an impltmcntation plan lor the OSFA quality improvement 
program. This chapter if)cludcs$ 

e An overview of the entire OSFA quality imp^vement program, wUb a 
special emphasis on the role of the corrective action component; 

o An implemoitation plan for the program; 

o A strategy lor implementing the corrective action rjonponent using the 
critical activities identified in Chapter 3| -6 

o Priorities for ongoing quality improvement. 
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CHAPTER I 
BACKGROUND 



This rtp»t p r «acnti the results of the Qtiality Control (QC) System Design 
Task, Stage Two Pelt of the Grant Quality Control Study. This task was initiated in 
an attempt to desi^^ a QC system for the Pell Grant Program. Based on the 
preliminary design of the Pell QC system and a comparison of the major student aid 
programs, the scope of the task was broadened to include ti*e Guaranteed Student 
Loan (GSL) and the Campus-Based (CB) programs, and to provide technical assist- 
ance in qualitative improvements in the delivery of these major student aid 
programs operated by the Office of Student Financial Assistance (OSFA). This 
refined i^jproach was based on th§ understanding tiwt there were many basic 
similarities in the delivery systems for these programs and that quality improvement 
was the responsibility of functional imits within OSFA. The quUity improvement 
program discussed in this paper can provide OSFA with a systematic process for 
identifying and correcting error-prone points in the delivery system. A major 
outcome of this design and technical assistance activity is a plan for an ongoing 
quality improvement program described in this final report. This chapter reviews 
the background and context for the d^gn study and disc»si» the meaning of 
quality control. 

THE PELL GRANT QUALITY CONTROL STUDY 

Quality, while often considered iU-deflned In many organizations, does not just 
happen. It can be realized only throqgh management programs that better utilize 
personnel and systems to improve product development and delivery. In a large 
social program, such as the Pell Grant Program and other student aid programs, 
mUlions of people are directly affected by the quality of the delivery system. The 
ultimate test of quality for a social program is whether the beneficiaries of the 
program receive the correct amount of aid, on time, and with a minimum ol error. 



M 10 



Addition^ly, from the pcrspcctivt of tht FtdtraJ government, it it important that 
aid be cieiivered ir as an effective and efficient a manner as humanly, or 
technically, possible. These factor»>-quaUty of the product and delivery system^ 
are of concern to po sts ec on dary instifjtlons, state agencies, and to the Federal 
government* 

Tiie three-stage Pell Grant Quality Control Stjdy was designed to provide the 
OS^A w^th a comprehensive examination of the status of the current delivery 
system. It was designed to incorporate the basic quality improvement prindpies. 
The quality control literature review discussed below provides three fundamental 
principles for quality improvement— prevention, identification, and elimination 
(PIE). These are generally characterized as follows: 

o P r ev en t i on refers to the design component of any production or delivery 
system. Prevention of error must be coraidered when a delivery pri>cess 
is designed originally, or in subsequent redesign; 

o Identification of error becomes important once a production or delivery 
system has been bnplemented. Typically, this would involve a statistical 
sample of products (awardees In the case of student aid) and the 
establi^vnent of measures and standards of rmitine error mcastrement; 

o Elimination refers to the process of correcting error in a production or 
delivery system once it has been implemented. This process is usually 
referred to as corrective action. 

The three-stage Pell Grant Quality Control StJdy, Illustrated in Figure I-l, 
provided a comprehensive quality bnproven^ent project. Stage One was a study of 
the quality of the current delivery system for Pell. Stage Two of the project was 
divided into three partsi Part I was targeted at designing and Implementing m 
ongoing measurement systeri for quality control in OSFA, Part n was an assessment 
of the effects of the current and alternative delivery system design (currently 
underway), and Part IH was an analysis of Stage One data. 

These three components of Stage Two closely parallel the PIE concept. Part 11 
focused on prevention by addressing basic long*range design problems with the 
student aid delivery system. Part m. Follow-on Analysis, is focused on continued 
and refined identification of errors In ttw delivery system. Part I, the Syston 
Design Compone n t, focused on the elimination of error in the current delivery 
system tfvough tiie ^plementation of an ongoing quality improvement program. 
This report presents the condusiorw of the Part I project. 
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FIGURE l-I 
PELL GRANT QUALITY CONTROL STUDY 
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Stage Thr««, currently underway, provide* a reitudy oi quality in the delivery 
system. This te the firtal report lor Part 1, Stage Two of the Pell Grant Quality 
Control Study. Af the figure iUurtratea, it draws irom other parts of Stage One and 
Stage Two, and provides a framework for continuing the OSFA quality improvement 

program. 

The Stage One Pell Grant Quality Control Study indicated 84jtostantial dollar 
error in awards to students durii^ the 19W-»1 academic year. Dollar error was 
defined as the actual award disbursements as listed in records at the sampled 
isidergraduate lnstituti*3ns in spring, 1981, minus what Advanced Technology 
calculated the correct disbursement to be using the best available information on 
application data, cost of attendance, and enrollmenr status. Total dollar error lor 
FY 1981 was estimated to be $275 per recipient, or $650 milUon of the $2.2 biUion 
(a 30 percent error rate) awarded to the 2.36 miUion recipients re,>rescnted by the 
sample. An estknated 71 percent of the recipients received an incorrect award, 
although in some cases the amount of incorrect award is quite small. Approximate;y 
percent of recipients had award errors in excess of $150. Net error (overawards 
minus underawards) was $<^02 million. Fifty percent of program recipients (or 
approximately 1.2 million students) received overawards totaling $526 million. 
Another 21 percent of recipients (approximately .5 million students) received 
underawards totaling $12* milUon. Stage Two moved beyond these basic research 
findings: 

o Part 1, summarized In this report, wa5 for the design and installation of 
an ongoing QC system for the Pell program; 

o Part n was an assessment of alternative delivery systems for Pell, GSL, 
and Campus-Based programs; 

o Part ni was a select set o* fcUow-on analyses using QC Stage One 
findings. 

The purpose of Part 1 was to design and initiate implementation of improve^ 
ment programs for OSFA. This part of the project was broadef than Stage I in two 
important ways. First, the scope of work actuaUy included providing technical 
assistance to operating units in OSFA. The technical assistance was provided, on a 
priority basis, to operating units responsible for qtjaUty improvement activities. 
This was a logical extension of the Initial design since the identification of areas in 
need of technical assistance was based t|>on an evaluation of the dtiiyery system 



BEST COPY AVAILABLE 



performed during Stage One ol the study. The technJctJ assistance activity actually 
took the lorm of assessing needs iot corrective action and making recommendations 

for specific correcthft actions* 

Second, the Stage D project, eapecialiy Parts I and U, included other Tifle^IV 
student assistance programs, particularly the Campus-Based Programs—the 
Su{>piemental Educational Opportunity Grant (SEOG), National Direct Student Loan 
(NDSL), College Work-Study Program (CW-S)-and the Guaranteed Student Loan 
Program (GSLP). The CSLP includes both Guaranteed Student Loans and Federally 
Insured Student Loam (FISL)* The basic goal of this task is to Improve the quality of 
student aid programs. The overall objectives of Part I of the Stage Two study were 
to design and Implement quality Improvement programs for OSFA through technical 
assistance aimed at designing corrective actions that would result in quaUty 
improvement in OSFA programs. 

THE MEANING OF QUAUTY CONTROL 

American govermncnt and industry have recently rediscovered the meaning of 
quaUty improvement and quaUty control. During the past decade, as problems 
emerged in the VS* economy, government and industrial leaders began to ask basic 
questions about the quality of production in the industrial sector and the quality of 
service delivery in govermnent. 

In the early 1970s, the Federal government faced the monumental task of 
going into production with a massive entitlement program the Basic Eckicational 
Opportunity Grant Program within a one year time-frame. This task was accom- 
plished successfully. Each subsequent year, the program grew in size and changed in 
some basic programmatic wry? consequently, the actual delivery of grants remained 
the critical issue. There wis little time to concentrate on the quality of the system 
used for delivery. The other student aid programs have faced similtt- problems with 
the need to deliver a profsnm that is continually being changed in the legislative 
process. For example, the GSL program has undergone major prcgrammatic changes 
through reauthorizations and technical wnendments of the Higher Education Act 
that have resulted in nearly constant modifications of the GSL delivery system 
during the past 12 years. Consequently, there is need for a systematic quality 
improvement program for the major student aid programs. 
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The QC literature provlde$ a framework ier a systematic quality improvement 
program. 3uran and Cryna (1970), two iaaders in the QC field, deline quality as 
fitness lor use. Accordinglyt QC should include activities which assure that 
products o»c:, services are lit. 3uran and Gryna emphasize that quality activities 
encompass the lile of a product, Iram design to post-<saie, although they recognize 
that often only a limited range ol these activities can be provided at any <me t^e. 
For student aid programs, this p er sp ective suggests that quality control should 
encompass the entire delivery process, Irom the application planning for student aid 
programs, to reconciling a cco unts after the aid is delivered. An Important 
distinction can be made between wpormd^ delects and chronic profatons. Ideally, the 
QC process should involve a breakthro ugh process to eliminate dronic problems, 
while sporadic errors can usually be eliminated through pr event i ve QC Figure 1-2 
illustrates the differences in the two ^iproaches to QC 

Historically, QC in the United States has emphasized technical approaches 
such as statistical QC, reliability, and product assurance. Increasingly, however, it 
is important for organizations to establi^ quality policy, with q>eciflc quality 
objectives that should be conveyed In written form. In his most recent work, 3uran 
«19S0 argues that two types of QC are necessary, managerial and technical. In 
building a management commitment to QC, a successlul QC analysis must consider 
the existing organizational responsibilities and the new design should be built around 
them. Additionally, a top management commitment to quality Improvement is 
necessary to maintain an ongoing quality improvement program. 

A variety of well-established tools and techniques are available for QC. The 
American Society for Quality Control maintains a publications program that covers 
such topics as national standards (197S} and guidelines lor managing vendors (19S0). 
There are well-established guidelines lor quality audits and quality cost analysis 
(Bajario, 1981), and well-defined plans lor establishing quality improvement 
programs (Kid well, 197 S). 

One especially uselul analytic technique in quality analysis is the Pareto 
principle which states that losses are never uniformly distributed over causes 
(quality or characteristics). Instead, losses are always unevenly distributed so that a 
small percentage, or a vital lew of the causes, contribute a higher percentage of the 
loss of error. This principle can be used to analyzes the distribution of loss due to 
error. The results of this approach are often a boon to managers and others 
concerned with instituting QQ it can facilitate an economical, targeted attack on 
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QC WITH EMPHASIS ON CONTROL 

I 

1. Choosing the control s(^ject->delinifig the i^iality, 
characteristic, or effort that must be regulated 

2. Choosing a unit of measured-defining the terms in 
which the control subject will be meafured 

3. Ctioosing a standard—defining the desired level of 
performance for ttie control subject 



H, Designing a sensor--creatlng a method of measiM'ing 
the control subject 

3. Measuring per|ormance~-perlormifvs fbe actual 
measurement 

6. Interpreting results-*-<»mparii^ the actual measure- 
ment to the standard 

7. Decision ma)(ing^<Hleciding on the action, if any, to be 
taken to the standard 



g« Action— taking tiM specific steps to bring performance 
up to the standard 



WITH EMPHASIS ON BREAKTHROUGH 



1. Breakthrough in attitudes— convincing those responsible 
that a change in quality level is desirable and feasible 

2. Discovery of the vital few problems—determining which 
quality problem areas are most important 

3. Organizif^ for breakthrough in knowlec^e— defining the 
organizational mechanisms for obtaining the knowledge 
for achieving a breakthrough 

4. Creation of the steering arm— defining and staffing a 
mechanism for directif^ the investigation 

5. Creatioi of the diagnostic arm-~def inlng and staffing a 
mechanism lor executing the technical investigation 

6. Diagnosis— collecting and analyzing the lacu required 
and reconunendlng the action needed 

7. Breakthrough In a cultural pattern— determinkig the effect 
of proposed change on the people Involved and finding ways 
to overcome the resistance to change 

S. Breakthrough in performance-staining agreement to take 
actton 

9. Transitkm to the new level— implementb^ diange 



ic 



FIGURE 1-2 
' BREAKTHROUGH VS. CONTROL 
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the bulk oi quality Imam. 3uran and Gryna (197p) argue that thi» simple truth, the 
Pareto principle, makes a quality improveinent program possible. Once the vital 
few problems have been Identilled they can be targeted lor management of 
systematic processes, identiiicatioo ol high error-prone areas, then targetin|^of 
technical assistance. These areas can be the basis of a systematic quaUty 
improvement program (KidweU, 1975), 

. These concepts are useful for student aid programs; however, the literature on 
QC is dominated ly the private sector, especiaUy industrial production. Most 
government standards have been generated by and for defense contractors. Conse- 
quently, the decision to institute the quaUty improvement programs for OSFA has 
been a one-ol-a-kind effortt it has applied basic principles developed in the QC 
field to the specific and unique problems 0i student aid delivery. 

THE OSFA QUAUTY IMPROVEMENT PROGRAM 

The overall goal of this task is to establish a framework for improving the 
quality of OSFA programs. A significant way to raise program quality is to 
routinize the process of QC QC, as used in this project, is tl^ process of 
preventing, identifying, and eliminating sources of error in a delivery ^ production 
system. The concept is most comprehensible when used in an Industrial getting. For 
example, it is easy to see the need for QC measures in the pi^duction of 
automobiles. If the quality of a particular make cf automobile varied greatly from 
car to car, the result would be expensive corrective action costs to the manufac- 
turer and decreased sales to consumers. Quality control is somewhat more 
ambiguous in a social service setting, such as the provision of student aid*, however, 
it is no less important. The Pell Grant Program, one of the largest student aid 
programs, illustrates this point. The annual overpayments in the Pell program, due 
to various institutional, student, and processor errors, are estimated in excess of 
$*00 million. This amount provides a great strain on progr»n resources and, due to 
annual funding ceilings, may reduce the size of the average award at the same time 
that education costs continue to rise* 

In both lr.<lustrial and social service settings, QC measures can increase the 
efficiency and effectiveness of the deUvery system by reducing costly errors and 
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raising the quality of services or products. In order to integrate QC into the 
everyday operations of an m'gamzation, at a minimum .* 

o Output standards must be established; 

o The established standards must incorporate technical specifications 
prescribed by the provider of the service or product and expected by the 
recipient or consumer; 

o A system for monitoring standards and correcting errors must be made 
c^>erat'imaL 

Among the possible operational procedires in a student aid specific QC program are 
the following: 

o Develop standards and measures for monitoring the delivery of student 

aid; 

o Measure performance of student aid delivery against spedlied measures; 

o Determine and monitor errors in eligibility determination and award 
processing; 

o Identify sources and probable causes of errors to plan corrective actions; 

o Develop corrective action prooeckres as an integral part of the process- 
ing functions; 

o Cevclop standards and measures for monitoring the results of corrective 
actims; 

o Ensure that various actors (e.g», i»t>oess<»-s or institutions) are operating 
in accordance with specified procedures, regulations, and standards; 

o Report appropriate QC information to Department of Elation 
personnel on a timely basis. 

In order to improve the quaUty of OSFA programs, two distinct procedures 
must be developed. First, an Of^oing structure or framework to determine sources 
of program error and to measure it must be developed. This component of a quality 
improvement program may u; called the teda^cal measurement component. 
Second, a formal mechanism lor designing and selecting procedures to eliminate 
existing error must be designed This component of a quality improvement program 
is called the corrective action co mp one n t. 



When thw« two componfits of a quality improvement program are integrated, 
they become a process lor maintainii* QC When viewed in the context of a quality 
control prooets, each compdnant has a series of well-defined subcomponents or 
steps. The technical measurement component includes the foUowtng steps: 

o Define the sampled siA>ject for control; 

« 

o Define a unit of measure; 

o Establish a standard of perlormanoe} 

e Create a measuring device or procedure* 

The steps in the corrective action component ares 

o Mobilize for measw^ent; 

o Compare actual performance with established standards; 

o Make management decisions on type of corrective action need d; 

o Implement corrective action. 

I 

An illustration of the interrelationships between these steps in a well-integrated QC 
process is shown in Figure 1-3. The QC process is illiwtrated as a cyclical procedure 
since the process is ongoing. 

The QC cycle was used as a basis for the design of the OSFA quaUty 
improvement program* The emphasis of this task was on the design of a technical or 
measurement component for the Pell Grant Program. The overall design for the 
OSFA Quality Improvement Program includes both measurement and corrective 
action components. 
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CHAPTER2 
GENERAL APPROACH 



This chapter reviews the general approach uMd in the design el the Quality 
Improvement Program. This program provides OSFA with a systematic methodology 
lor identifying, measuring, and correcting errors in the student aid delivery system. 
The chapter has three sections. The first reviews generic approaches to quality 
improvement, including the strategic approach which was used lor the task* Second, 
alternative QC system configurations for the measurement co m ponent of the OSFA 
QC system are reviewed (this analysis was used by OSFA to target the technical 
assistance phase of the task). Finally, the actual framework used in the task is 
summarized. The general approach pr e s e nted in this chapter is A tested methodol- 
ogy for improving quality In student aid programs. 

GENERIC APPROAOiES TO DEVELOPING QUALITY COHTROL PROCEDURES 

The quaUty control process should not merely be an afterthought or an 
addendum to an existing system. Unfortunately, the provision of student aid does 
not represent ^s ideal situation. Althoui^ some c o n cern has been paid to qiMdity 
control procedures, no rigorous and methodical quality control process paralleled the 
development of the delivery system. Therefore, quality control procedures must 
subsequently be integrated Into an operational delivery system. Three generic 
approaches have been identified for introducing quality control procedures into an 
existing delivery system. The first is characterized as the incremental bottom-up 
approach, the second as the oo mp rehens l ve approach, and the third as the strategic 
or modular approach. 

The incremental bottom-up approach to quality rontroJ development typifies 
the approach used in most g o vernment agencies, educational institutions, and 
industrial settings. This approach assumes that fisv i subunits within an 
organization have responsibility for their own corrective actions and as a result, no 
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syitcm-wide quaJJty control plan ttdsts* Ttitrtiort, quality control development is 
usually 8|>oradic, incremental, and seldom the outcome of coherent planning. 
Further, since corrective actlom are identliled by subunits, quality control problems 
involving interface with other subunits are rartly rasolved. The typical i^^ps 
followed by an organiTstien takii^ an in c rem e nt al approach to quality control artt 

0 FunctioTta^ :;*iounlts ((fiviston* branch, etc.) discover error>prone areas 
through problems with the system in operation, 

o Plant for corrective acttois (new prooadtn^ or system changes) art 
developed and based on needs as they arise— usually a limited range of 
options are oonsideredi 

o Managers seek new VMuroes for corrective actions— system develop- 
ment or implementation of new p rooe d ire s — but probl«ns that involve an 
interface with other subunits are often ignored unless the resource issues 
can be resolved; 

o Quality control |M'ooe<fcires are developed and implemented only as time 
and resources permit. 

The Incremental approach has dominated quality control development in 
student aid. While the Division of Quality Asstrancc (DQA) has implemented some 
quality control reporting requirements in the past, there have been no systematic 
attempts to develop quality control procedures. An incremental approach to quality 
control is likely to produce some serious problems. These include: 

o Completion of specified tasks in the necessary time frame without 
concern for smooth toleration and reckiction of error in the system and 
efficient use of personnel and othe- resoiffces; 

o Inconsistency and variation in the ways functional subunits deal with 
quality control problems; 

o Acute problems in the quality control of products that cut across 
functional subunitt (or that involve more than one Division/Branch). 

The comprehensive ifiproBCh assumes ^t anytiiing that can go wrong with ttm 
delivery system will go wrong; therefore, it is important to identify every possible 
error in the system and design corrective procedures. In order to introduce quality 
control procedures Into the student »id delivery system according to this method- 
ology, it would be necessary to identify all program subsystems and all the major 
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•eters or components In tach st^ytttm and th«n define a rasponsive term ol 
corrective actions. Specifically, the basic steps in a comprehensive approach to 
quality controJ are: 

o Identify the ma)or suteystcms of the programs (t'Os includes pre- 
application, application, eUglhlUt) determination, benefit calcuUtion, 
fund disbursement, and account reconciliation); 

o Identify the major actors or com p one n ts for each subsystem (for 
example, f or the PeU aUglbiUty siAsystem, actors include students/ 
IMrents, institutions, and a^piUcation processors); 

o For each actor In each subsystem, identify acceptance measures of 
possible errors in the system; 

o Define meamm far each set of standards including identification of 
data elements and procedures for information collection; 

o Determine ^ co mp one n t s of each siAsystem that merit development 
and inclusion in the quality control system and evaluate the feaslbiUty of 
Including tye various information sources in a quality control data base; 

o Proceed with system development on the select subsystems (design and 
develop procedures for implementation of selected components of each 
scft>system)* 

The strategic approach assumes that the major sources of error in student aid 
programs can be identified and corrective action should be made in these areas 
according to « step-wise or modular methodology. Thus, to utilize thb approach, 
significant sources of error must first be identified, prioritised, and then corrected 
in a hierarchical manner. The basic steps in the strategic approach aret 



o Conduct a functional malysis of the operating system, indudiM infor- 
mation raquiremants, linkage structures, and breakdown points. In other 
words, identify the places in the system where corrective action can be 
taken and monitor p r ogr e ss; 

o Identify significant sources of error in the program; 

0 Select md priorltlM targets for systematic QC development and the 
time frame for spedfl^tions, design, development, and installation of 
each select measurement systems; 

o Proceed with systenrts development for selected measurement systems 
(e.g., develop procedures and systems manuals, user manuals, system 
sp£:ifications, and software specifications, as necessary); 
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o Pftriorm iystcm t«st$ on selected measurement tystems as they are 
cofTipleted* 

Since the Stage One study identified major problem areas in the delivery of 
student aid, the project team and OSFA decided that an incremental approacfT to 
quality control was not adequata. Both the comprehensive and strategic approaches 
to quality control were viable stratagiea, however. 

For the Pell Grant Program, the comprehensive approach would result in an 
elaborate quality control system that imposed new data collection procedures on top 
of the already existing systr^r*. It would permit the development of a separate or 
stand-alone Quality Control Management Information System (QCMIS). It could also 
be used to produce quality control manuals for training ED p er s o nn el (Central and 
Regional) and institutional representatives in quality control prooedires. To the 
extent that the system used automated data collection and analysis procedures, it 
would be labor intensive since virtually an entire new set of procedures would be 
needed for each component of each subsystem. Also, sophisticated data base 
management procedures could be needed centrally, depending on how much of the 
system was actually implemented. The com pr ehensive approach was considered too 
ambitious for implementation. Trying to implement the system all one time would 
bring down the entire delivery system. 

The strategic approach to quality control development in the Pell program 
would permit the incremental implementation and testify of modular quality control 
subsystems designed specifically to reduce errors in the system. The modular 
approach could also permit the use of up-to-^te electronic tech«iology milizing 
preexisting data sources, where appropriate, rather than developing entirely new 
dau sources and reporting prooeckires. It might also result in more systematic 
analysis and reporting on data currently collected. Some new data collection would 
invariably be required, however. The strategic approach provides OSFA with a 
flexible approach to targeting resources on areas of greatest need. The risk 
associated with this approad) is that some imporuuit area could be overlooked, and 
consequently, a major problem could go unattended. 

After analyzing the two approaches the project team and DQAa decided to use 
the strategic approach. The basic trade-off between the comprehensive and 
strategic approaches is comprehensiveness versus timeliness. Since there is an 
immediate need for quality control procedures in the student aid process and since 
funding is tight in all government programs, the project team recommended utilizing 
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the strategic approach to introduce quality control procedures. This approach has 
the greatest potential lor: 

o Reducing errors in the Peii program; 

o Adding other student aid programs to the quality control process on an 
of^o&ng basis; 

o Pilot testing quality control components earlier in the study; 

o DeveJc^ing a sound responsive methodology to corrective actions. 

ALTERNATIVE QC SYSTEM CONFIGURATIONS IN OSFA 

Two important design issues conoeming the QC system or program are: (I) the 
degree to which the system will be automated; and (2) the degree to which existing 
reporting and information processing can be utilized. These issues were considered 
early in the design process as part of the conceptualization ol the quality 
improvement program. ; 

The issues of automation and report formats can be used to generate five 
feasible configuration options. 

o Option h a QC system providing manual er^ianoements to the existing 
delivery system and requiring new reporting formats. 

o Option 23 ; a combined manual/automated QC system requiring new 
reporting formats. 

o Option 3$ a combined manual/automated QC system using existing 
reporting formats. 

o Option a fully automated QC system using existing reporting 
formats. 

o Optional a fully automated QC system reccing new reporting 
formats. 

Existing data are not of sufficient quantity or (^lality to mnkt providing manual 
enhancements to the existing system and using existing requirements a viable 
option. Each option can be compared using the following evaluation criteria: 

o Feasibility of the system design (Can it be done?); 
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Q Potential lor reducing error (Will it save money?); 

o Deveiopmemal costs (How much?); 

o Interface with the itelivery system (U'iU it work?). ^ 

Option U Manual Enhanoenicnts 

it is possible to develop a quality control MIS that essentially provides manual 
enhancements to the existing Pell delivery system. This MIS will require develop- 
ment of detailed report lormati for each of the major actors in the financial aid 
delivery system. The reports would be entered into a filing system In the Division of 
Quality Assurance and used as a means of monitoring and tracking progress on 
certain key quality control areas* Cither the comprehensive or the strategic 
approach to quality control system development could be used to develop the manual 
enhancements, although the comprehensive approach is easier to adapt to this 
option. The major problems with this q>tion are that it would add to the reporting 
burden and introduce excessive time delays in program monitoring due to its 
nonautomated format. 

Option 2t Combined Manual/Automated MIS Requiring New Reporting Formats 

This option would essentially take the approach in Option I and where 
appropriate, automate data files* Other flies, including periodic summary reports 
using aggregated data, would remain manual. This option has the potential for 
providing data on a somewhat more timely basis than Option 1 but would provide an 
additional layer of rq>wting on top of the existing delivery system. It could use 
either the comprehensive or strategic system development approach but would be 
more editable to the fcrmer. 

Option 3s Combined Manual/Automated MIS Using Existing Reporting Requirements 

This option would have some of the same features as Option 2 but would 
emphasize new analyses of existing data soirees rather than development of entirely 
new reporting formats. This aj^oach would be flexible enough to add other student 
assistance prc^sns as necessary. In this way, a series of QCMIS subsystems could 
be constructed that dealt with critical poinu in the delivery system. This option 
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would be most adaptable to the strategic approach for quality control system 
development. 

Option #s FuUy Automated/Integrated MIS Uains Existing Reporting Formats "'•^ 

This option would take the approach in Option 3 to the fullest possible degree 
of system automation. Such an option would integrate the core QCMIS with systems 
that interface with the major actors* operatli^ systems. Moriit(»-ing, comparing 
results to specific s tandard s, taking routine actions, and rtportii^ could be built into 
the system. Such an approach would be de p ende n t on automation of most 
components of the student aid delivery system. This option could use a variation of 
either the strategic or the co mpr e he n s ive approach to systems development but 
would have to be done in combination with the redesign of the entire delivery 
system. 

Optkm 5t Fully Automated MIS Requiring New Reporting Formats 

Option 5 is identical to Option 4 except existing data would not be considered 
sufficient to create a functional MIS. Thus, additional reporting burden would be 
added ior actors in the delivery sysmm. 

Assessment 

Figure 2-1 summarizes the preliminary assessment of the iive generalized 
optim against the evaluation criteria. On the basis of the preliminary assessment, 
it is possible to make an initial jud^ent about which QCMIS configuration option is 
most desirable. 

Option 1 would probably have relatively modest results on reducing error, 
moderate developmental costs, and would not create an integrated QCMIS delivery 
system. Option 2 would increase the potential lor reducing error but would raise 
develc^ental costs. Option 3 has high potential for reducing error, would require 
moderate developmental costs compared to Options 2, and 5, and would be 
partially integrated into the delivery system. Options ♦ and 5, while having the 
highest potential for reducing error, do not appear feaswie at the present time. Of 
the live options. Option 3, a combined automated/manual system using existing data 
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aourcxs, ^ipears to Im» the moet <Mrabte and ylabk QC wystem configuration. It 
would create a QC systm that is mource savlf^, usir^g the enhancing existing data 
and management practices, rather than resource fkabitnKt facing new demands on 
the current delivery system. 

FRAMEWORK FOR DEVE' OPING QUALITY CONTROL PROCEDURES 

Based on this analysis, a strategic or modular approach to the development of 
^lality control procedures for the DSAF student aid programs was utilized. It 
adapted the various steps in the strategic approach to (piality control development. 
These steps can be broken down into two phases— design and implementation. y'The 
basic steps in the design phase weres 

o Identify significant soirees of error in the program (completed during 
Stage One); 

o Conduct a functional analysis of the operating system, including infor- 
mation requirements, linkage structures, and breakdown points. In other 
words, identify the places in the system where corrective action can be 
taken and monitor progress; 

o Conceptualize a quality contrcl system with modular components 
designed to detect and monitor error-prone functions. 



The steps in the implementatkm phase were; 

o Select and prioritize modular subsystems for development. Also, 
identify time frame for specifications, design, development, and installa- 
tion of each selected si^system} 

o Proceed with systems development for selected subsystems (e.g., develop 
procedures and systems manuals, user manuals, system specifications, 
and software specifications, as necessary); 

o Perform system tests on modular subsystems as they are completed. 

During the design phase, the focus was on a QC measurement system for the 
Pell Grant Program. As a result of a functkmal analysis and a comparison of Title 
IV Programs, suggesting many similarities between programs, all three major 
programs were included in the basic design. As a result, the preUminary design was 
expanded based on a comparative analysis of the Title IV programs to provide a 
basic framework for the overall OSFA (piality improvement program. 
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The impStmtntatiQn phase was characterized by technical assistance rattier 
than detailed system design per se. This was undertaken in recognition that OSFA 
divisions must be responsible for quality improvement in their own areas of 
responsibiiit). The technical assistance was targeted on error-prone areas *^t 
needed QC enhancements and were designed to provide an ongoing management 
report. 
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CHAPTER 3 
OSFA QUALITY IMPROVEMENT PROGRAM 

During the past three yean, OSFA has initiated an internal quality improvement 
program. This program has wed the strategic approach to quaUty improvement to 
identify and correct error-iNm acthrittes in the current deUv«ry system. This effort 
has focused om 

o Identification of opportunities for quaUty improvement in the overaU 
delivery sjrstem 

o Development of an overall strategy for improving the quality of the 
current delivery system 

o Implementation of internal mechanisms for improvii^ student aid delivery. 

This chapter describes the results of this effort and presents the long range plan 
for improving quality in the student aid delivery system, using the general approach 
discussed in Chapter 2. The chapter has four sections- First, the opportunities for 
quality improvement are discussed. Second, the overall strategy for quality improve- 
ment is reviewed. Next, the framework for the internal quaUty Improvement program 
is reviewed. Finally, the steps in implementation of the overall strategy are discussed. 

OPPORTUNITIES FOR QUALITY IMPROVEMENT 

The development of the overall quaUty bnprovement strategy in OSFA was based 
on a detaUed analysis of specific opportunities for quaUty ^provement. A three-level 
framework was used to assess ^mUty control needs of OSFA. Figim >-l presets an 
information jy$tem«8 perspective of OSFA based on this framework.^ The three levels 



are: 



The poUcy-level analysis examines the type of quaUty control informatkm 
required by entities that interact with OSFA in setting policy for student 
aid. These actors indude remainder of the Department of Education (ED), 
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the Office of Management and Budget (OMD), and Congressional staff. 
Interactions between OSFA and these entities usually occur as part of tc 
the anmtftl budget prooan or legislative raauthorizatlon. 

o The management-levai analysis reviews the organization of OSFA jnd its 
current management procedures related to quality control. The aaalysis 
includes an evaluation of the types of quality control information proce- 
dures re^iired lor the elfectlvt management of the Pell program. 

o The delivery-level malyals considers the functional role of each organiza- 
tional entity within OSFA« Examined are the quality control procedures 
currently in place in each entity, the information resources within the 
enUty^ and the information needs of the entity. 

At the policy level, quality control concerns relate primarily to overall program 
deUvery. Outside of OSFA, In ED and OMB, there is a concern that payment error in 
tne Pell program, for example, should be assessed on an annual basis. Additionally, 
analysts in the Office of Planning* Bu(%et, and Evaluation (OPBE) are interested in 
receiving timely analysis of policy options for corrective actiois that require 
legislative or budget action. Tbsse svne concerns are apparent within OSFA, along 
whh an accurate awareness that critical parts of the policy cycle (plannii^ budgeting* 
and regulation) and program delivery process (application ttMwigh reconciliation) must 
be performed on a timely basis. 

At the nianagement level, quality control has been hindered by the lack of 
information on the quality of system performance. The fuhcttonal organization within 
OSFA makes programmatic Improvement more cfifficult, especially when activities in 
one Division require information and actions from other Divisions. The lack of 
information about error-prone points can limit the ability of managers to track critical 
activities and make improvements. The timely flow of information between functional 
units is often limited, in part due to the quality of the overall information system. 
Too often the information needed by another Division, regardless of its importance, 
takes second pfiority to performance of key activities within the Division. As a result, 
there is a need for an overall strategy that identif es critical error-prone points in the 
delivery system and provide a framework for implementing corrective actions, 
especially as they relate to the flow of management information within OSFA . 

At the delivery level, existing quality control prooecfcsies are more abdmdant. 
Occasionally, Branches and IMvislons hsve developed ^leir own quality control 
processes, usually on an ad hoc basis. As a result, there is wide variation in the 
awareness of staff within OSFA about quality control issues. Some Branches have 
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devtlopad lalrly sophistlcftttd ap p f o a eh t to quality control while others are ftUl 
plagiMd by basic probl«ms* Unlortmteiy, none ol these procedires are tied together 
in an overall system with a well defined ilow of iniormation* 

As part o! this pro)ect, OSPA s ponsore d an exhaustive study of the ef iects^Bf the 
oirrent delivery syitem for the PeU^ CSL, and Campus-Based programs.* It revealed 
that the most seriously negative effects of the current system for the Federal 
government are the unavaUahlUty of information and lack of fund oontroL^ Most of 
the causes of these negative effects can be improved through marginal changes to the 
current system. For example, most of the basic problems with fund control for 
student aid programs can be corr ect ed by improvements to internal accounting 
procedures. For example« in the CSL program, where fund control is the most severe 
problem, changes in accounting procedures can ameliorate many of the deficiencies 
identified by the General Accounting Offioe.^ Orim marginal changes for GSL, such 
as increased use of state quarterly reports in quality control checks for state claims 
and collections activities, could also reduce fund control problems. 

Quality control measurement can play an important role in the overall strategy 
for delivery system improvement* Using the stratcgk approach to QC system 
development, it is possible to put quality control checks into place for critical 
activities throughout the student aid delivery system. Such development provides 
senior policymakers with the early warnings that they need about major system 
problems, as well as provides program managers with the detailed information they 
require to develop strategies for correcting delivery system deficiencies. For 
example, «fciring the past year the CSL Branch of the DWislon of Program Operations 
put into place a new system for measuring error In the manual interest payment 
proc e ss. This enabled OSFA to find errors before ^y were uncovered by end-ol-year 
audits and, in turn, to make needed correcttons. 

The importance of improving the ^lallty of information about the financial 
management of Federal student akl programs is echoed In the findings of the 
President's Private Sector Survey on Controh Task Force Report on Education. ^ The 
task force identified the need for improved management information systems and 
internal controls as a mechanism for reducing waste, fraud, abuse, and error The 
report considered ttm need for ^»ecific improvements in student aid programs and 
delivery, indudii^ improved debt collection. However, a major emphasis of the report 
was on the improvement of management information and fund accountability, two 
closely related problems. 
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Short of sn wcm dtUvtry i^om rcdtslgn, which takes substantUl resources 
and several years to implement. Improved quality control holds the most promise for 
addressing these basic problema. Marginal improvements can resolve many ol the 
basic problems with the student aid delivery system. Quality im p r o veme n t eK erts 
should be specifically targeted on Improvements in fund control and avallabliity of 
program information since these areas represent some of the most basic problems 
facing the Department. 

THE OVERALL ^^TRATECY 

OSFA hn developed an overall strategy for quality improvement that can be 
implemented on an incremental baab and that targets resources on the most critical 
problems. The overall design of the quality control program is iUustrated in Figure 
3-2. The overall strategy has lour major co mp on en t s* 

o Institutional Quality Control, which encourages institutions to make 
delivery system Improvements; 

o External Quality Control, which provides quality assurance functions for 
institutional QC and provides overall measures of erron 

o Internal Quality Contr^, which Is designed to facilitate marginal improve- 
ments to tiW current systemi 

o A Quality Control Management Information System (QCMIS) that uses 
inputs from the ab^rr^ dted sources to tailor reports f on 

Policymakers requiring corrective action analysis, early warnings of 
system problems, and summary error reports 

Managers concarned about klentlfyfaig specific QC targets, tracking 
critical activities, and monitoring overall system performance 

Program personnel requiring basic routine information to improve the 
delivery process and financial accountability. 

Each of these comp eiie nts of the overall design for the OSFA i^ty control 
program is discussed briefly below* The internal component is discussed in more detail 
in the next section* 
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Policy Level 
QC Reports 

o Corrective Action Analysis 

o Early Warnlf^ Reports 

o Summary Error Analysis 
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Mmagement Items 
QC Reports 

o Identify Critical QC Targets 

o Track Critical Activities 

o Monitor Sjrstem Performance 



Delivery Level 
QC Reports 

o bnprove Quality of Information 

o Improve Efficiency 

o Improve Productivity 



FIGURE 3-2 

DESIGN FOR 05FA QUALITY CONTROL SYSTEM 
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Institutknal Quality Control 



It is the %oa\ oi OSFA to encourage the deveiopment oi systematic internal 
quality control, through measurement oi discrepancies and errors in the dotivery 
system, in all postsecondary institutions participating in OSFA student aid programs. 
OSFA is taking two actions to facilitate this type of development. 

First, OSFA is considerii^ regulatoc7 changes that would require postsecondary 
institutions to sample Campus-Based rodpient records to measure error. The 
objective of this requirement would be to encourage institutions to measure errors in 
their own delivery system and to develop corrective actions if they have high error 
I rates. Ovcf the long term, OSFA will require institutions to measure and correct 

discrepancy of errors lor all Title IV programs, not just lor Campus-Based. 

Second, thro^gh the **target of opportuntty** approach initiated during Phase n of 
this project. Advanced Technology developed a ham&ook outlinii^ quality control 
procedures for the financial aid office. A draft of thh cbcument was reviewed by 
members of the student aid community. Currently, the handbook is being revised to 
reflect OSFA and community concerns about long-term quality improvement at the 
institution level. This emphasis woidd place responsibility for quality control with the 
instituticm. 

External Quality Control 

Until institutions can implement a comprehensive instituti<»ial quality control 
plan, h is necessary for OSFA to develop and maintain a systematic process for 
measuring error In the entire delivery system. This is called external quality 
c<mtrol.The external component of the OSFA Quality Improvement Program focuses on 
the three major program areas: Pell, GSL, and Campus-Based. The Pell Quality 
Control Study, conducted by Advanced Technology and Westat, Inc., was a compre- 
hensive study of error in the Pell program. A systematic framework for replicating 
the Pell Study was developed, and currently is being implemented. The Campus-Based 
segment of external quality control can be implemented either independently, or in 
combination with the Pell Quality Control studies. 

The measiresnent of error in the delivery of the CSL program presents special 
problems that could not be handled through modification of the Pell Quality Control 
studies. Since CSL involves a complex network of lenders and guarantee agencies as 
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well as imtitutions, appUcanU, «ncl applicant families, it will be necessary to uke a 
different emphaais for this study. It would, at a minbnum, include: 

Measurement of applicant error throufh a method similar to the Pell-Rudy; 

Measurement of institutional error throqgh methods similar to the Pell 
study; 

Measurement of guarantee agency and lender error through an entirely new 
approach. 

The goal of OSFA Is to reAice its role in direct external measurement onoe an 
institutional quality control plan is operationaL At that time, OSFA's role wiU become 
one of quality assurance. Quality asswanoe for external measurement is the process 
of monitorii^ the effectiveness of bwtitutional quality control programs and determin- 
ing institL«tional compliance with ED quality control regulations. 

Internal Quality Control 

The long range goal of the internal quality control program for OSFA is to 
develop QC measurement and reporting mechanisms for critical activities throughout 
the student aid delivery system. During Stage II, Part 1 of the Pell QC Study, the 
following activities were selected for technical assistance using the strategic 
framework: 

0 OSFA Goals «id Objectives System 

o GSL Manual Interest BiUlng 

o FISAP Processes for Campus-Based Programs 

0 External Quality Control System 

o Vendor/Processor Quality Control 

o Institutional QC Guidelines. 

Subsequently, additional targets have been added. One such target initiated 
during Phase m deals with the GSL reinsurance process. 



o 
o 

o 
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Quality Control Management Information Systam 



The major itatatt oi the entire QC system is a quality control management 
inlormation system (QCMIS) that receives inputs from ail of the above quality wtfol 
subsystems and can be used to generate reports for all levels d management. 
Conceptually, the QCMIS will include a framework lor routine reporting from all other 
measurement subsystems, internal and external, as n^eli as special analyses commis- 
sioned by the Assistant Secretary as part of the corrective action framework, which is 
discussed in the next chapter. Most of these reports can be generated by the other 
modules. The QCMIS should consist of a defined set of information flow and reporting 
specifications lor each of the modules. 

The framework for the routine reporting from other measurement modules to the 
QCMIS Is presented in Appendix A. This framework includes: 

o Summary reports from the external measurement module; 

o Stmmary reports from the vendor/processor (^ity control and goals and 
objectives modules; 

o Sample n^xxt formats for the internal QC mockiie. 

The quality control management information system, when fully developed, will 
provide management reports for managers throi^hout OSFA. It is the objective of the 
Division of Quality Assurance to continue with the development of the QCMIS 
framework, as outlined in Appendix A. 

INTERNAL TARGETS WOSi QUAUT/ IMPROVEMENT 

The purpose of the internal OSFA quality improvement program is to establish 
mechanisms for measuring critical error-prone points in the delivery system and 
improving the way these activities are accomplished, if necessary. The section 
presents an analysis of critical activities in the student aid delivery system. The 
analysis recognizes the following features of the Federal role in the delivery system; 

o There are more similarities than differences in the delivery systems for the 
major student aid programs, which suggests that an integrated approach to 
QC is desirable. 
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o Some aitical activitiw are ^^st addressed at a management level in OSFA 
since they art acreis the entire delivery system. 

o Other critical activities relate specifically tc Individual subsystems and 
progranuh. 

First, there are similarities across the delivery systems ior student aid programs 
by design. The Office of Student Financial Aid has a functional organisational 
structure that combines activities for many functions. Very often the same personnel 
work on all three programs, the current delivery system for the Pell, GSL and 
Campus-Based program has six subsystemss 



o Pre-^ipiicBtion, which refers to the pro-am plamiing and IxKlget develop- 
ment, information dissemination and other activities that usually take 
place prior to the start of an award year. 

o Student application^ which refers to the actual processing of student 
applications. 

o Student rligfhiiity determination, which refers to the process of determine 
ing categorical student aligibiiity and need for each program. 

o Student benefit caicuiatlon, which refers to the process of calculating and 
packaging the awards. 

o Fund Disbursement, which refers to the process of disbursing fwxfs from 
the Federal government to students, institutions, guarantee agenci<», and 
lenders. 

o Account ReooncUlatlon, which refers to the process of reconciling all 
accounts — for students, institutions, lenders, states and the Federal 
government — after the award year. For GSL, this is by definition a long- 
term process since the Federal gover nm en t subsidizes interest on loans 
after they are made. 



For the purpose of internal quality control in OSFA, the three student systems • 
application, elegibility determination, and benefit calculation — can be 'fi'eated a 
single subsystem. The Federal government has relatively little direct involvement in 
these subsystems, except for the PpH program application process. Instead, campuses 
and, to a lesser extent, lenders and guarantee agencies are the primary actors in these 
subsystems. 

Second, there are some critical activities in the delivery §ystem that cut across 
all stdisystems and are most ^>propriately addressed at a management level in OSFA. 
Two of these critical activities ware addressed during Stage n of the Pell Grant 
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QuaUty Control Study, One U vendor /processor quality cwitrol and quality assurance. 
Private contractors play an Important rote in each subsystem. As a result, vendor/ 
proeeisor quality asstirance cuts across subsystems. One oi the technical assistance 
activities In Stage U, Part was the development of guidelines ior vendor/pracessor 
quality control lor large processor contracts in OSFA and the development of a quality 
assurance manual ior project officers lor these contracts. 

• Another activity that dearly cuts across all sid>systems is the Goals and 
Objectives System in OSFA. This system is used to monitor all deUvery system 
activities for each progr»n. Durii^ Stage H, Part I ol the QC study, the Goals and 
Ob|ectlves System was also selected lor technical assistance. In this area, the 
network approach to program management was pilot tested, and found applicable to 
the OSFA program management. This api»*08Ch has not yet been implemented. 

Third, there are numerous activities in the deUvery system that are candidates 
for quality control and quality improvement that are most appropriately addressed on 
an activity-by-activity basis within subsystems. The purpose ol this section is to 
identify the critical activities using a Knmd overall framework. Critical activities are 
delivery system procedures that are particularly error prone. The framework used to 
identify critical activltiess 

o defines the subsystem 

o identifies the goals of thit subsystem 

o identifies the quality control objectives for the subsystem 

o identifies the relevant quality con^ol measures 

o identifies critical activities lor quality control that correspond to these 
goals, objectives, and measures. 

In the remainder of the section, the framework is applied to the four key 
subsystems in the student aid delivery. These are: 

o Pre-application Subsystem 

o Student Application, Eligibility Determination, Benefit Calculation Sub- 
systems 

o Funds Disbursement Subsystem 
o Account Reconciliation Subsystem 



3-11 



Pre- Application Subsystem 



The pre-appllcation subsystmi to the prcx^ss for out year program planning. It 
includes all acthrities related to deveiopment of forms, budgets, and a^Ucation 
information. The quality control goal of this suteystem is to conduct all pre- 
application activities in an efficient, tknaly, and responsive manner. The critical 
Federal activities at the delivery level in the pre-appiication subsystem are identified 
in Figure 3-3. The reasons for including these targets arc considered below. 

Of the targets already selected by OSFA, only the two management level QC 
targets — vendor / proce s s o r quality assurance and the goals and objectives system — 
explicitly address the pre-appiication subsystem. Vendor processor quality control 
relates to pre-application since the processor contracts, such as Pell, must be changed 
each year to incorporate new requirei^cnts, which is a process that should be 
monitored through vendor processor quality assurance. The Goals and Objective 
System explicitly acknowledges the planning (or pre-application) cycle, as well as 
delivery and wrap-^> cycle of any program year^ 

The development and promulgation of new regulations is often viewed as an 
impmant area of concern by the stiident aid commtmity. The major problem, with this 
activity is the timing of new regulations relative to the time and costs of implement- 
ing them in the field. If there is ample lead time, campuses are more likely to be able 
to adjust to new regulations as part of a routine planning process. However, when new 
regulations come out just fX'icM' to or durii^ tiie award year, tiie prcd>abiiity of error is 
irnrreased due to the fact that adjustments must be made at the last minute, if they 
can be made at all. A QC analysis of this activity i^XHild be done from the per^>ective 
of the impact on the entire delivery system. Key measurement points could be 
idmtified that would provide OSFA managers and policy makers insight into possible 
problems before they happen* 

The devel<^>ment and revision of forms is also a critical factor in filling out the 
right information at the right time. The Coals and Objectives System partially address 
the forms development process, since it is identified in this system. However, the 
impact of delays in these key forms has not been systematically analyzed. It is 
possible to develop a mechanism for monitorif^ ihc timeline and availability of , 
important forms. 
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PRE-APPUCATION 



The process that includes iarccasting and developing budgets, developing and 
promulgating federal regulations, developing necessary icHins, disseminating 
program information to participants, training participants, contracting^and 

Elanning for scrviots, determining institutional profsram eiigibliity, estaoXish- 
tg payment systems, and planning for progron specific procedures. 

Coal: 

# 

Conduct all pre^application activities in an efficient^ timely, and responsive 
manner. 

Quality Control Obfecthm 

Timely development of regulations 
Timely promulgation of regulations 
Timely development and printing of forms 
Adequate availability of informatkn 
Timely information dlsMmlnation 

Timely and accurate responses to telephone and mall inquiries 
Timely determination of initial Pell authorization levels 
Accurate determination of initial Pell authorization levels 
Timely determination of Campus-Based allocations 
Accurate determination of Campus-Based allocations 

Quality Control Messurass 

Ntfnber of months from finalization of regulations to beginning of program 
year 

Number of months from prcxnulgation of regulatims to beginning of program 
year 

Number of months from development and printing of forms to beginning of 
program year 

Number of months from forms development and {H-inting to begiming of 
program year 

Number of months from information dissemination to beginning of program 

Comparison between participant information needs and availability oi infor- 
mation that is disseminated 

Accuracy of answering participant questions by monittM-ing telephcHie calls and 
sampling mail responses 

Number of days to complete initial authorization for Pell 

Difference between initial authorization and final allocation in Pell 

Number of days to complete authorization for Campus-Based 

Percent of Qunpus-Based allocatims that are appealed 

Difference between tentative allocation and final allocation in Campus-Based. 



FIGURE 3-3 CRITICAL ACTIVITIES IN THE 
PRE-APPLICATION SUBSYSTEM 
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Activiti«ft for Quality Controb 

e ED development and promulgation of new regulations lor each program 
as required 

o ED development of lorms including application forms and instructions, 
authorization letters, rtquests for payments and reimbursemenU, SASls, 
progress reports, validation rosters, FISAP, loan assignment forms, and 
teacher cancellation forms 

o ED dissemination of program information through dear colleague letters, 
participant training, responses to participant inquiries, and deveiopmem 
of handbooks and manuals 

o ED initial authorization of kmitutional funds activity for Pell by DPO 
through the Pell Disbursement System 

o ED tentative through final allocation of institutional f uncb for Campus- 
Eased programs through FISAP proce»ing in DPO. 



nCURE 3-3 CRITICAL ACTIVITIES IN THE 
PRE-APPLICATION SUBSYSTEM 
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Another critical prt-appiic»tion activity the dittcmimtion of program Inior- 
mation through various sources* There was substantial critidim of OSFA c&iring the 
recent public hearii^s that Important inlormation is simply not available when needed 
and that when it is available it is otten oontracfictory. For exanple, state gudfantors 
who operate in diiierent states have observed that dllierent regional offices interpret 
est regulations differently. During site visits, campus financial aid administrators 
observed that they often have to make numerous calls to OSFA to get «iswers to 
simple questions about the Federal end of the delivery system. This problem cuts 
across Divisions In O^A and» therefore, should be addressed at the management leveL 

The initial authorization of funds for the Pell program is another critical activity 
that can be tracked throMgh the quality control system. This activity sets an initial 
allocation which can cause major problems for the institution if It is too low and cause 
an excess of float if it is too large. A coherent and systematic approach to monitoring 
this activity could be developed within the Pell Grant Branch, DPO. 

Student Application, Eligibility Detenninaiion, and Benefit Calculation Subsystems 

The Federal gove r nmen t is not directly involved in most of the activities 
included in these subsystems. Therefore they were combined for this analysis. These 
subsystems represent the actual Interface between the student and the delivery 
system, which takes place at several points. The Federal government is only directly 
involved in this subsystem for the Pell program. The quality control goal of this 
subsystem is the timely processing of applications and the accurate determination of 
eligibility and benefits. 

The critical activities for this program are identified in Figio-e 3-4. These are 
the processing of applications, determination of eligibility and benefits for students 
applying to ADS institutions, and validation procedures used to verify selected 
application data items. All of these activities have already at least been partially 
addressed by the quality control study. 

The processing of applications for Pell was treated explicitly in the report of 
vendor/processor quality control Procedures for improving quality control were 
actually built into the new Pell contract. 

ED deter .nination of eligibility for students attending ADS schools is the only 
delivery system activity that involves a direct interface between OSFA and students. 
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STUDENT APPUCATION, EUGIBILmr DETERMINATION, AND BENEFIT 
CALCULATION 

The proctn by which t itudent applln ior iinancUl aid either through & 
processor or institution, the processor or institution reviews the application 
tor compliance with eligibility requirements, and award amounts or maxiilj^ 
loan amounts are determined. 



Cottk 

Timely processing of student applications and accurate determination of 
eligibility and benefits. 



Quality Control Ob)ectiveas 

Timely friitlal Pell processing 
Timely Pell corrections processing 
Reliable Pell processing 

Accurate Pell categorical eligibility determination 
A :curate Pell benefit calculation 
Accurate Pell validation 



Number of days between application receipt and mailing of SAR 
Number of days between corrections receipt and mailing of corrected SAR 
Percent of edit identified errors by Pell processor that are actual errors 
Number of transactions pa- applicant 

Percent of applicants ruled eligible who actually meet eligibility criteria 

Percent of benefit calculations computed accurately 

Percent of eligiblliity determination errors remaining after validation 



Activities lor Quality Controls 

o Processing of student applications, calculation of SAI, generation of SAR 
by PcU processor* 

o ED determination of eligibility and benefits for students applying to ADS 
institutions* 

o Validation procedures to verify SARs used by schools in RDS and by ^ 
in ADS, 



* Activity already at least partially addressed. 



FIGURE CRITICAL ACnVITIES IN THE STUDENT 
APPLICATION, ELIGIBILITY DETERMINATION AND BENERT 
CALCULATION SUBSYSTEMS 
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Numtrcus quality control proct<kr«i, including wmpling and error me^ Airemcnt, can 
bo implemontod for thii activity, Just as thoy can be implemented at the campus level. 
T»ie campus level quality control guidelines that are currently being developed iof 
OSFA can be adapted to this tunction. The validation procedires used by schoeib for 
RDS and ED lor ADS is also addressed in the institutional quality control guidelines. 

Fund Disbursement Subsystem 

Fund disbursement is the process of disbirsing program funds from the Federal 
government to state agencies, tenders, institutions, or students, and from institutions 
to students. The quality control goal of the funds disbursement subsystem is to 
transfer the right amount of money to the correct recipient at the proper time. 
Critical activities are identified in Figure 5-3. 

QuaUty control procedures have already been developed for two activities in the 
funds disbursement siOJsystemx interest and special allowance payments in GSL and 
(iisbursement of Campus-Based fimds throqgh the FISAP. QuaUty control procedures 
for interest and special aUowance payments monitored the accuracy of disbursements 
from ED to lenders and guarantee agencies by examining the efficiency of the receipt 
control process for 799 forms, the completeness of submitted 799 forms, the accuracy 
of calculations on the 799 form, whether dupUcate payments were made to lenders or 
guarantee agencies, and the accuracy of Treasiay vouchers authorizing payment. 
Processing timeliness was also monitored by ensuring that the date on the certification 
letter or Treasury memorandum for a voucher preceded the penalty date for a 799 
form. This process has helped identify problem areas In the imercst payment and 
special allowance |M-ocedures in need of corrective action and allows subsequent 
monitoring of the impact of the corrective actions. 

Quality control work for FISAP concerned developing corrective action 
strategics for error-prone functions. Analyses were conAicted of the most common 
errors committed in FISAP reporting and problems with FISAP procedures and forms. 

ED disbursement of funds for the PcU program to RDS schools is a critical 
delivery system activity. It takes place as an integral part of the Pell Disbursement 
System. During the past year, the PeU Grant Branch of DPO has made several 
improvements in the accuracy of the process. The purpose of the quality control 
model would be to monitor the timeUness and accuracy of this process and to report on 
the financial implication of this process. 
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FUND DISBURSEIilEMT 



The payment of program funds from the federal government to state agencies, 
lenders, institutions* or students, and from institutions to students. 



Coals 

Transfer the right amount of money to tiie oxrect recipient at the (roper 
time* 

Quality Control Objecthrces 

Tfanely disbursements 
Accurate dbrtMreements 
Minimize float 

Accurate in«yeer progrsn cost estimates 



Quality Control Me asu r es 

Number of days between scheduled disbursement date and actual disbursement 
date 

Difference bet w een actual disbursement and "correct" disbtrsement 
Percentage of disbursements in error 

Amount and timing of dlsburacment to institution compared to amount and 

timing of dl^Mrsement to student 
Difference between actual program costs and estimated costs for program 

year 

Activities for Quality Controls 

0 ED disbursement of funds for the Pell program to RDS institutions and 
RDS students tlvough the Pell Disbursement Syrtem 

o ED disbursement of funds for the Campus-Based program to institutions 
through the FISAP process* 

o ED disbursement of funds for the Pell pro gra m to students in ADS 
institutions through the Pell Disbursement P r ocessor 

o ED payment of interest to lenders participating in the GSL program 
while student borrower is enroUed, in grace period, or is in deferment 
period and ED payment of special allowance to lenders to subsidize 
Guaranteed Student Loans* 



* Activities for which control has already been at least partially addressed* 



RGURE 3-5 CRITICAL ACTIVmES IN THE FUND 
DISBUR^MENT SUBSYSTEM 
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£0 payment of administrative cost allowances to guarantee agencies 
participating in the GSL projr^m to compensate them for servicing costs 

Reftmds in Pell program funds from institutions to ED for students who 
graduate early, withdraw, or drop below half-time enrollment during the 
t^e covered by the grant, 



FIGURE 3-5 CRITICAL ACTIVITIES IN THE FUND 
DISBURSEMENT SUBSYSTEM 
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The ED disbursement of funds to schools in the Campus-Based program happens 
as A drawdown proem* This activity is most appropriateiy adcfa«sscd as an accounting 
issue since it currently has no niechanism, except audits, for determining if funds are 
being cfrawn down according to Federal regulation&« The ED accounting systeiajioes 
not currently track these requests on a program basis. The routine tracking of this 
activity could ivsult in significant cost saving due to the tightening of the float for 
in»titutions that draw down funds sooner than aUowed by Federal guidelines. 

ED disbursement of funds to ADS students is primarily an issue for reasons of 
timing. The Stage One PeU Quality Control study revealed that the actual disburse- 
ments for the ADS students arc relativeiy accurate. However, the timing of 
disbursement can be problematic for ADS students and instltutkms. A quality control 
mechanism for this activity could monitor both timing and accuracy. 

ED payment of administrative cost allowances to guarantee agencies In the GSL 
program is another critical activity. ED currently has no mechanism for monitoring 
the accuracy and timeliness of this disbursement. It is possible that data reported to 
ED on quarterly reports could be better used for this inirpose. 

The collection of refunds in the PcU program is 3 critical activity th^t is 
difficult for ED to monitor. When there arc changes in student enrollment status, this 
usually means that a refund is due to the Federal government, since most changes are 
for reduction in course load. Corrections to the SAR are usually not reported unti. the 
end of the year in the 5VR. Analysis of this activity could focus on QC enhancements 
to the Pell Disbursement System. 

Account Reconciliation Subsystem 

The account reconciliation subsystem includes all processes required to reconcile 
program accounts for a given award year and collect loans. The QC goal in account 
reconciliation process is to elkninate incorrect payment throD^h record reviews and 
audiu and reduce loan default rates. The account reconciliation process provides a 
back end quality control for the entire system. 

The critical activities in the Accoi»rt Reconciliation Subsystem are identified in 
Figure 3-6. None of the quality control technical asistancc provided dwing Stage 11 
directly addressed the account rcconcilliation subsystem. It is, therefore, a subsystem 
with several opportunities for quality improvement for internal quality control. 
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BEST COPY AVAILABLE 



ACCOUNT RECONOUATION 

The process oi reviewing records and determining that the amount of program 
funds difbursed to each student» institution, lender, or guarantee agency is 
correct, and, the process oi student loan repayment. ^ 

Goals 

Eliminate incorrect payments throqgh record reviews and audits and reduce 
* loan default rates. 

Quality Control Objectivcss 

Accurate recapuav of ADS sUident overpayments in Peil 
Timely recapture of ADS student overpayments in Pell 
Accurate recapture of institutionai overpayments in Pell and umised funds in 
Campus-Based 

Timely recapture of institutional overpayments in Peil and unused f und^ in 
Campus-Based 

Accurate information on enrollment status for students with FISL loans 
Accurate payments by lenders and guarantee agencies to ED for cdlectims on 
defaulted loans 

Timely payment by lenders and guarantee agencies to ED for collections 
defaulted loans 

Accurate recapture of overpayments on ciiiims for defaulted loans 
Timely recapture of overpayments on claims for defaulted lo4ns 
Accurate information on borrowers teaching in low-income schools or in 

military or •'Head Start" service 
Timely collections on NDSL loans assigned to ED 
Correct collections on loans assigned to ED 
Accurate Institutional audit and reVtew procedures 
Accurate guarantee agency audit and review procedures 
Accurate lender audits and reviews 
Accurate call reports and lender manifests in FISL 
Timely call reports and lender manifests in FISL 

Accurate quarterly reports, acfrninistrative cost allowance letters, and tape 
<kimp for GSL 

Timely quarterly reports, acfrninistrative cost allowance letters, and tape dvmip 
for GSL 

Quality Cqptrol Meastress 

Difference between ADS student overpayment and £0 collections 
Number of months between end of award year and closing student account 
Difference between institutional overpayments and ED collections 
Nisnber of mwiths betwe«m end of award year and cl<»ir^ institution account 
Elapsed time between change in era-oUmcnt status and receipt of information 
by ED for FISL borrowers 

Difference between actual lender or agency collections on defaulted loan and 

amount reported to ED 
Percent of collections on defaulted loans reported to ED in error 



FIGURE 3-^ CRITICAL ACTIVITIES IN THE 
ACCOUNT RECONCILIATION SUBSYSTEM 
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Number of days from lender or agency collection to transfer to ED 
Percent of claim over^yments recaptured 

Number of months ffvm identllication of claim overpayment to recapture of 
overpajinent 

Percent of borrowers whh cancelled loans for which employment status data 
are accurate 

Percent of loans assigned to ED upon which collections are made 
Percent of Institutional auditors and reviewers foilowing required fnrocedures 
Percent of guarantee agency auditors and reviewers following required 
procedures 

Presence of required data elements on call reports and lender manifests 
Percent of lenders meeting ED r^>orting requirements for call reports and 
lender manifesto 

Number of days between due date of call reports and lender manifests and 
submission tete 

Completion of rec^dred data elements on quarterly reports, ac^inistrative 

cost allowance letters, and tape chsnp 
Percent of guarantee agencies meeting ED reporting requirements lor 

quarterly reports, administrative cost allowance letters, and tape disnp 
Number of days between due date of quarterly reports, administrative cost 

allowance letters, and tape dump and si^ission date 

Activities lor Quality Controls 

o Student account reconciliation lor students enrolled in ADS institutions 

o Institutional account red»^Ulation "dirough verllication of student 
validation roster in Pell and through FISAP in Campus-Based 

o Enrollment status reporting to determine if loan is eligible for interest 
subsidies lor FXSL 

o Recapture of overpayments to lenders and state agencies on defaulted 
loans and capture of collections made by lenders and guarantee agencies 
on defaulted loans 

o NDSL cancellation when a borrower is teaching in a low-income school 
or is in military or "Head Start" service and assignment of NDSLs in 
default for two years to the federal governm«it for collection 

o Institutional audits by independent auditor and program review by ED 

o Guarantee agency audits by independent auditor and program review by 
ED 

0 FISL lender audits by indq>endent auditor and program review by ED 

o FISL lender reporting through call reports and lender manifests 

o Guarantee agency reporting through quarterly reporu, administrative 
cost allowance letters, and tape dump 



nCURE CRITICM. ACTIVITIES IN THE 
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ADS account reconciliation lor student accounts is one of the aiticaJ activities 
in the account raconcUiation mAtymm, The objectives of a quality control 
enhancement for this activity iwouW be to develop a routine measure of the difference 
between the actual disbursement and what should have been disbursed. ^ ^ 

Another critical activity is reconciliation of institutional accounts tor Pell 
through the Pell Disbursement system. The problem with this activity is delays in the 
reconciliation process, possibly due to the absolute standard used for reconciliation. A 
QC study of this activity could locus on improved tolerance levels and standards for 
the reconciliation process. 

A doeely related activity is reconciliation of iristitutional accounts for Campus- 
Bank programs through the FISAP process. The verification of FISAP can be 
approached through detailed analysis of consistency on the FISAP form. ThU has been 
done on an ad hoc basis in the past by ths Campus-Based Branch, DPO. A routine 
sampling procedure could be used. 

Enrollment status reporting to determine loan eligibility for interest subsidies is 
critical for FISL and GSL. For FKL, the timing of requests for enrollment verification 
has a built in problem lor the Federal government? the bi-annual reporting for 
enroilmem leaves too little time between verifications for students who drop out right 
after the report is made. For GSL, enrollment reporting is an extremely compUcatcd 
process of concern to lenders, GAs and institutions. A number of quality improve- 
ments are possible in this prooef« . The default rate for GSL and FISL could be reduced 
through such an effort. 

For the claims and collections process in GSL and FISL the government lacl<$ a 
systematic approach to check the reasonableness of claims. Therefore, the Federal 
government currently lacks the capacity to correct on over payments. This is an area 
where an enhancement study would result in substantial savings. 

NDSL cancellation and assignment of NDSL to the Federal govuTiment is 
another activity that has not had quality control checks in the past and where 
improvements are possible. A QC enhancement in this area could include a systematic 
sampling of cases to determine overall error rates. 

Institution audits by independent auditors and program reviews by ED is another 
activity in which quality improvement could result in savings. While a school 
monitoring system for the DCPR review process is in the design stage, it has not been 
implemented and ED presently lacks the capacity to do analysis of sources of error. 
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This information couid be used iof all aspects of OSFA management, from regulation 
development threi%h a oca un t ctoMOUt. 

Guarantee agency autfits asW program reviews is another activity where quality 
improvement could possibly lead to financial savings. ED currently Jacks the systems 
auditors needed for thoroi^h program reviews for GAs. A QC study in the area could 
more precisely identify the review needs lor CAs. 

The £D review of lemlers suffers from a lack of a sound information system that 
can be used for analysis purposes. While lending institutions are generally highly 
regulated, there is now a real possibility that when lenders do submit a bill with errors 
it will go undetected by the Fecieral government. A CSL QC study may be necessary 
to define and measure lender error to proWde a benchmark far the audit and review 
processes. 

Guarantee agency reporting is mothtn' activity where quality imorovenient is 
needed. CAs report though quarterly r«^>orts, administrative letters and state tape 
dumps. The quarterly reports couid be better utilized by OSFA for QC checks on 
various aspects of the GSL reconciliation process. The state tape dump also offers 
possibilities in this area, but has not been consistantly reported to ED. The quality of 
data on all three reports has- not been routinely checked by OSFA. The entire OSFA> 
state interface is an area where quality ImfM-ovement is needed and a QC enhance* 
ment study is possible. 

THE NEXT STEPS 

OSFA has made substantial progress during the past three years in the design and 
implementation of an overall quality imiM-ovcment prc^am. The emphasis of the 
program durir^ the first two years has been on the measurement component. During 
Sta^e One of the Pell Quality Control Study the emphasis was on error measuremem 
for the Pell program. During Stage n the emphasis shifted to internal quality 
improvement within OSFA. During Stage Three, OSFA has maintained a dual emphasis 
on internal quality imjix-ovement md external measurement of error in Pell. 

During Stage m of the Quality Control study, a strategic approach to internal 
quality improvement was developed, tested, and used to select high error prone 
targets. This chapter has reviewed those targets, identified overall strategies for 




quality improvement and identilijied new targets ior quality improvement. Possible 
next steps in this internal quality Wprovtmtnt process are tot 

o Select additional targets lor technical assistance 

o Develop procedures lor internal QC development by operating units in 



o Develop the QCMIS reporting system. 

First, it is now an opportune tfane lor selection of additional targets ior quality 
improvement. This chapter has presented a set of critical delivery system activities 
that can be selected ior technical assistance during Stage Thnt of Quality Control 
Study. Work so far during Stage m. Part n, of the QC study, h&s included GSL 
reinsurance and refinement of the institutional QC guidelines. 

Second, the 05FA should also consider development of guidelines for overaU 
quality improvement in OSFA. A framework for this activity is considered in the next 
diapter. 

Finally, OSFA should also consider development of a QCMIS for routine reporting 
on all quality control and quality improvement activities. The framework for this is 
presented in Appoidix A. 
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FOOTNOTES 



^Adapted Irom R.N. Anthony. Piaming »nd Contro) Systiyns: A Framework for 
Analysis ^ (Boston, MA: Harvard Business School, 1965). 

^S-C. Blumenthal, ManagCTnent Inlormation Systems (Engie Woods *tiliffs, 
N3: Prentice Haii, i%9h 

Advanced Technology, Inc^ and Westat, Inc. Assessment of Alternative Student Aid 
Delivery Systems? Ass««nent of the Current System^ prepared for the Credit 
Management Board, ED, and the Division of Quality Assurance, OSFA, 3une 
1983, pg. 3-13. 

^Ibid, pg. 3-U. 

^The Guaranteed Student U>an lirformation System Needs a Thorough Redesign , U.S. 
General Accounting Office, September 2k, 19S3. 

^Svtoitted to the Subcommittee for Consideration at the meeting on 3une 13, 19^3. 
^Ibid, pg. viL 
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CHAPTERS 
IMPLEMENTATION PLAN 

This report has described the results of Stage n. Part I of the Pell IQC Study, a 
des;«,ti and technical assistance effort, and has focused on the measurement 
component of the OSFA Quality bnprovement Progrvn. In order to adiieve the 
overall objective oi this project, to design and implement an ongoing quality 
improvement program, OSPA must continue this systematic quality improvement 
eif ort. in order to facilitate the process, this concluding chapter considers: 

o The overall Quality Improvement Program design; 

O The implementation of the Quality Improvement Progrem; 

o The development of a corrective action component; 

o Priorities for action. 

OSFA QUALITY IMPROVEMENT PROCS^ 

OSFA has made a concerted effort over the past three years to Identify and 
implement quality ccmtrol measurement and to organize corrective action analysis. 
The Pell Grant Quality Control Project has proposed a wide range of corrective 
ctions. These proposals have not adequately involved OSFA personnel, nor has a 
formal structure for corrective action l>een proposed previMJsly as a result of the 
current project. This section considers the basic elements of a generalized quality 
improvement program for OSFA. 

The quality improvement program has two basic components; the technical QC 
system with an emphasis on measurement, and the analytic component with an 
emphasis on corrective actions. The major emphasis is in the design task on the 
technical measurement component which attempts to identify error-prone points in 
the financial aid program. Having identified these points, the objective of the 
corrective action component is to introduce program reforms which can increase 



Qverall program quality. The steps in tht technical measurement and corrective 
action components, as well as the interaction l>etween these components, is 
illustrated in Figure Quality control has been defined as a process of 

identifyingf correcting, and preventing error or a tendency toward error in a system. 
Therefore, as shown in Figure 4-1, a formal ongoing quality control effort must 
combine both a technical (or meaaurament) component and an analytic (or 
corrective action) component* 

The technical com p onent of the quality control process is already well into the 
design stage. Figure ^1 shows that the technical component must include a 
capacity to: 

o Define samples; 

o Define measures; 

o Establish standards; 

o Develop measurement mechanisms; 

o Implement measur«n«it mechanisms. 

The analytic or corrective action component of the overall quality control 
procedure is essential to close iht loop and repeat the quality improvcsncnt cycle on 
an ongoing basis. The critical elemmts of the correct!^ ^ion component, as 
shown in Figure <^-l, are: 

o Comparison of actual performance with standarcb; 
o Management selection of corrective action options; 
o Implonent corrective actkxis; 
o Repeat the cycle. 

The development of a formal corrective process in OSFA would require signi- 
ficant changes in OSFA management. Some of the assumptions that were 
considered in the development of this framework were; 

o The ongoing quality improvement program, especially the corrective 
action component, should be integrated into the overall management 
system In OSFA; 
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0 The corrective action framework must interlace with other management 
systems and procedures in OSFA, including the Perlormance Monitoring 
System iPMSSi 

0 It is important that the corrective action component provide OSFA 
personnel with opportunities to initiate and receive recognition ior 
corrective actions Initiative, increase<^oductivity, and error reduction; 

0 A management commitment to quality in OSFA is essential to the 

implementation of the overall QC system; / 

/ 

/ 

o The implementation of the corrective ictfon component of the Quality 
Improvement Plan should be an integral part of the ongoing QC system. 

When proposing that a formal organizational mechanism be established for the 
ongoing quality improvement program, it is necessary to consider the organizational 
intent in which the system will be implemented. In OSFA, features of the current 
system should be recognized in the overall design. These include: 

o The formal management structtre in OSFA and current initiatives to 
improve management; 

o The placement of the new system in the organizational hierarchy. 

The Office of Student Financial Assistance is organized into functional 
divisions with responsibility for different aspectt of the delivery of all three 
programs. Divisions are further subdivided into branches. In some divisions, such as 
the Division of Program Operations and the Division of Policy and Program 
Development, there are separate branches for each major program (Pell, GSL, and 
Campus-Eased)* > In others, brmches are divided by function. For example, the 
Division of Certification and Program Review has five branches, ead) with responsi- 
bility for different functions. In the larger diylsions, there are sections and units 
with further refined sets of responsibilities. Within this hierarchical structure, 
OSFA is in the process of knplementing several significant mvtagement enhance- 
ments. 

One of the major management enhancements that is currently being imple- 
mented in OSFA is the Coals and Objectives System. This system identifies goals, 
objectives, activities tasks, sid>tasks, and steps for ihe delivery of each major 
student aid program. This §yst9m has the potential of strengthening the manage- 
ment of individual programs as a complement to the functional management system 
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that It curr«ntJy in operation* It Mentilies unltt and indivi<k»als r«sponsibie itx 
completing individual stepft* Currantly, OSFA Is exploring a networking approach to 
the Coais and Objectives System which will improve its program management 
capability. 

Another significant management enhancement being implemented in OSFA is 
the Performance Monitorii% Sjrstcm (PMS). PMS will provide branches and sections 
in OSFA with a formal mechanism fors 

o Identifying performance measures for individuals, units, sections, and 

o Reporting on routine performance of work activities; 

o Establishing goals for improving performance within units; 

o Monitoring performance of individuals, units, and sectiof^ 

o Evaluating performance of individual employees based on established 

criteria; 

o Recognizing and rewarding exceptional performance* 

This system involves emplo3fees in establishing criteria and setting perform- 
ance goals. It is being implemented in branches where the work activities are of an 
ongoing repetitive nature, such as forms preparation or review. 

Both systems will provide OSFA with an improved management capability. 
However, in spite of these iimovations, the management structure in OSFA remains 
a hierarchical structure with a top-down information flow about policy and 
procedures, and a bottom -t^ flow of information about work performance. Conse- 
quently, there is a gi^ between policy formulation and actual work activities in 
OSFA, a gap accentuated by the absence of information about the types of actions 
that can be taken to improve error-prone areas and functions in the delivery system. 
This relationship is depicted graphically in Figure 4-2. The quaUty control process, 
with its technical and analytic co mp one n ts, is intended to give OSFA a formal 
mechanism for closing the gap. 

The quality improvement program is intended to provide OSFA with technical 
and analytic support for the overall OSFA quality improvement process. The overall 
quality improvement process must be well integrated into the management structure 
of OSFA, as well as provide a mechanisn for closing the gap between policy 
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formulations and organizational activities. An illustration suggesting the placement 
and the role ol the corrective action process is presented in Figure 4-3. This 
placement and role is explained more lully in the iollowing discussion. 

The basic design of the quality improvement process with its technicaT ^nd 
analytic components, should enhance the roles of OSFA managers in instituting 
corrective processes in their units* 

9 

IMPLEMENTATION OF QUAUTY IMPROVEMENT PROGRAM 

The design of the OSFA <|uallty improvement program must consider the 
formal organizational mechanisms required to put the framework into action. Four 
formal mechanisms are critical to Implement the overall quality control process and 
the corrective action framework* These are: 

o Assign ResponsibiUty— an individual or group must coordinate the correc- 
tive action process; 

o Develop Proee^ra for Initiating Corrective Actions— procedures must 
be developed to give OSFA personnel the opportunity to gain recognition 
for identifying corrective action options; 

0 Develop Procedures for Impienoenting Corrective Action— a formal 
mechanism for implementing new procedures related to the corrective 
actions component of the quality control process must be developed; 

0 Develop a Reporting System Interface QCMIS— the corrective actions 
framework must be integrated with the technical aspects of the quality 
control system. 

Assign Responsibility 

There arc two optional approaches for formalizing the quality improvement 
program in OSFA The first would simply require appointing a senior official to be 
responsible for the implementation of the overall program in OSFA. This official 
would designate staff responsibilities for implementing the process and for working 
with divisions and tM-anches. 
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Alternately, the Deputy AstUtant Secretary ior Studef»t Financial Assistance, 
in conjunction with Division Directors and Branch Chlels, could appoint an OSFA 
Quality Council, The memttership oS the council, In combination, might include: 

o Representatives irom each of the OSFA Divisions; 

o Individuals with responsibility for the areas identified earlier as targets 
lor increasing program quality and reducing error in each Division or 
Branch; 

o At Jeast one representative from each program, GSL, Peil, and Campus- 
Based. 

The Quality Council should be responsible for ^ overall quality control 
process, both the corrective action component and the technical component. The 
overall size of the group should be kept between 10 and 15 people, if possible, since 
larger groups are more difficult to convene and manage. The Division of Quality 
Assurance should provide staff assistance to the Quality Council. The purposes of 
this council should include; 

o Responsibility for the OSFA quality assurance function; 

o Approval of Brand) and Divisim qualitv control plans (with particular 
emphasis on cross-divisional implications); 

o Responsibility for implementing the corrective action process and devel- 
oping OSFA policies and procedures for implementing and initiating 
corrective acticms. 

Corrective Action Procedures 

Another important mechanism for the overall quality control process will be 
the development of a procedure for establishing corrective actions. This procedure 
should be established by the Quality Council or senior official in charge of 
corrertiye actions. . _ _ — — 

When establishing procedures for initiating corrective actions, the Quality 
Council or responsible individual should recognize the different types of corrective 
actions that can be taken. Specifically, the council should distinguish between: 

o Type 1— Working level corrective actions that can be implemented at the 



spot where the worked is performed. Usually, the Section Chief or 
Branch Chief can approve this type of corrective action. The Quality 
Council or responsible individual should be concerned primarily with 
reporting ol these actions to the QCMIS as a formal mechanism for 
monitoring marginal changes; 

0 Type 2— Cwrec ti ve action analysis should be required for marginal 
changes that affect more than one division or have implications for the 
overall delivery system for one of the programs. The Quality Council or 
responsible individual may reserve the option to appro re the selected 
option. In such cases, Hhe corrective acticm anal)rsis should consider 

effects of the current procedures on key partidpanu, 

effects of options on key participants, 

selected corrective action; 

o Type ^— P rogr am levd corrective actions should be analyzed by the 
Quality Couricil or responsible individual. Again, ^ effects of the 
current system should be analyzed along with the differential effects of 
the marginal chaise optlms coraidered. Recommendations ^Knild be 
submitted to the Deputy Assistant Secretary for approval; 

o Type ^Policy tevel corrective actions inually require actions outside of 
OSFA. Some marginal changes, such as forms redesign or simplification, 
require policy decisions at the level of the Secretary and above. In these 
cases, the Quality Council or responsible individual should submit recom- 
mendations to the Dqwty Assistant Secretary. The Quality Council or 
reponsibie individual should, consider the types of additional procedures 
that would be necessary for Type 4 corrective actions. 



Prooediff«s for Implementing Corrective Actiors 



Whenever a corrective action has been approved, new working procedures are 
necessary. It is possible that tiiese are never doosnaited. Very often, formal 
written procedures will be necessary. For example, when the GSL Brar«ch, DPO, 
undertook corrective actions in the area of manual interest billing, the branch 
instituted new procedures. At the very least, the Quality Council or responsible 
individual shcnild eitabli^ an everi^ proc e d ur e for Implementing corrective actions. 
This should include: 



o Descriptiwi of the corrective action; 

o Description of activities' tasks and steps affected by the change (perhaps 
related to the Goals and Objectives System); 
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o Documentation oi written procedures that are to be changed; 
o Notation oi new procedures that should be developed. 

QCMI5 Reportii^ 

The corrective action system should feed directly into the quality control 
ma'nagement iniormation system. The corrective action reports to QCMIS should 
consist oii 

o Summary reports irtmi Divisions and Brianches; 

o Corrective action analysis reports; 

o Corrective action implementation reports. 

These reports will vary in frequency and purpose. Stsnmary reports should be 
designed to provide periodic reporting on progress. Along with the implementation 
of the corrective action process, the reports could provide the basis of the 
corrective action reporting system. The type of corrective action analysis reports 
used would depend on the types of corrective actions being implemented according 
to the above framework. The corrective action implementation reports would 
provide a mechanism for reporting on the effects, or savings, of implementing each 
corrective action. Formal report formats could be developed for each type of 
report. 

CORRECTIVE ACTION COMPONENT OF OSFA QUAUTY IMPROVEMENT 
PROGRAM 

The development of the corrective action cwnpwient of the ongoing quality 
improvement program would represent a significant departure from past practice in 
OSFA. It would require iHiilding a commitment across OSFA to quality 
improvement. As discussed earlier, this would require a formal d-signation of a 
senior OSFA official or other pe'son to be responsible for implanentaticm of the 
quality improvement program, especially the corrective action compwwnt. The 
implementation of the ccrrective action component would require implem»iting a 
corrective action process. 



The senior oilicial appointed by the Deputy Assistant Secretary, or Quality 
Council, should initiate an annual quality knprovement program In OSFA. This 
requires a signUicant investment in analysis* 3ohn L« Kidwelll observes: 

A quality improvement program is an investment; manpower will be 
spent 11 results are expected. In order to put everything into the proper 
perspective, a fact-linding activity is a prerequisite to this investment. 
One good way of involving the whole organization in this phase is to 
give key managers spedllc responsibilities in the fact finding. The 
recommended approach is throi^h an appointment of an ad hoc team. 

Kid well recommends that the ad hoc team have the same membership require- 
ments as the Quality Coimcll recommended in the previous section. The basic steps 
proposed by KidweU for the annual quality knprovement analysis plan are: 

0 Determine quality policy and currmt complianoe with quality policy; 
o Determine or estimate quality costs; 
o Identify dominant quality problems; 

0 Determine compliance to the (^>erating units quality system; 

o Identify the existing defect prevention system; 

o Collate and analyze findings; 

o Develop recommendations for unit management. 

These steps are used here as a basis for an acticm plan for implementing the 
OSFA corrective action system. The action plan is outlined below. 

Determine Quality Policy and Complianoe 

Determination of the OSFA quality policy could be one of the most critical 
tasks of the Quality Council. Kidwell defines quality as "that degree of excellence 



hohn L. Kidwell, A Profit Plan for Quality , Waterford, Ct.j The 3<^ L. Kidwell 
Company, 1975. 

2lbid, p. 34. 
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of a procKict or service that provides lor iuU customer satisfaction, over the 
expected life, with timely availability at a cost to the customer that he can afford, 
and at a profit to the procedur«8**«3 Qearly, this definition would have to be 
modified to fit a definition of quality for student aid delivery systems. It is possible 
that the QC objectives in each subsystem, identified in Chapter 3, could serve as a 
basis for developinig the OSRA cpiality statement. Once the definition is developed, 
the' Quality Council initiates the action plan. The first step in the action plan is for 
the Quality Coimcil to request Division Directors to: 

o Review OSFA quality definition and program requirements; 

o Determine the applicability of the definition to their organization; 

o Determine whether or not improvements in the operation are required to 
adopt tl'ie quality policyi 

o ' Determine the extent to which curroit <k>cumentatiQn am* procediires 
are adequate. 

Divisicms and Branches may identify significant im|rovements that should be 
made. For larger Branches, the Branch Chief may need to involve section chiefs and 
unit chiefs in the program. The informatioi generated from this process will 
provide a starting point iar developing the c(M-rectlve action system. 

Estimate Quality Costs 

Costs of Quality (COQ) is a concept Kidwell recommends for highlighting and 
displaying the "cost of imquality." According to this view, the concept of quality 
cost management is a simple one— *'once you know these costs, you can take steps to 
reduc| those costs that offend ycHi**.* In order to acWeve this type of incentive 
structure, it is necessary for the units to repwt: 

o Costs of quality program (either a QC module or system enhancements 
develq>ed internally); 



3lbid, p.30. 
*Ibid, p. 36. 



o Costs of losses caused by nonconiormance to standards. (This will 
require f uUer specification of loss or gains.) 

This type of analysis can help Division and Branch Chiefs to identify areas 
where corrective actions can be implemented. For example, during the past year, 
both the Pell Branch and CSL Branch of DPO have Implemented marginal changes 
that could result in substantial savings. The establisi jnent of this type of reporting 
system would provide a formal mechanism for giving recognition for such enhance- 
ment. 

The specific critical activities Identified in Chapter 3 can provide a basis for 
this step. The focus of this preliminary analysis should be on identifying the amount 
of error in the system. 

Identify Dominant Pr o M c ms 

The basic question here is, *^hat needs to be fixed?** Quality costs tell 
managers the areas where improvements are needed. The Pareto approach to 
corrective acticm analysis can be applied. This aj^oadt recognizes ^t 80 percent 
of the problems are caused by 20 percent of the cases; therefore^ the Division and 
Branch managers should focus on their most error-prone areas. They should be 
asked to identify their own dominant quality problems, their seriousness and 
magnitiKle. They should be asked to separate the Vital few** problems from the 
"trivial many.** Corrective actions should be directed toward important problem 
areas. 

This list of critical activities also provides a basis for this step. OSFA 
managers, or perhpas the Quality Council, ccKild go through this list to identify 
priorities ior oxitinued action. 

Determine Compliance to Unit Quality System 

This will be a two-step process. Since most units now have an overall quality 
improvement plan, it will be necessary to first develop a quality plan for the unit— 
this should evolve out of the review in the prior step. 
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G«nerally» organiiation* have two qu&Uty systems: The one they think they 
have, and the ones they actually have«^ In order to determine the actual quality 
systenf), it will be necessary to ask OSFA managers: 

o li the hasic functions are being perlmmed; 

o If these are established standards for these activities; 

0 

o It these standards are actually adhered to; 
o U the new standards are needed* 

Kidweii recommends that the Quality Council establish a "Quality Audit 
GuideUne." Once these guidelines are est^Ushed, several quality audit teams 
should be formed to the selected areas. The audit teams should have the 
responsibility in the area belr% audited. 

In OSFA, audit teams should work on those activities that are considered the 
highest priority. Technical assistance may be desired for some high priority targets. 
The emphasis should be on establishing and refining reasonable tolerance and 
standards for these critical activities* 

Determine the Defect Prevention System 

Defect prevention, especially marginal corrective actions, should be the 
responsibillTy of operating units* The corrective action procedures outlined in the 
previous section should help establish this principal in OSFA. The Quality Council 
should consider whether the operating units have internal corrective action systems. 
The basic question that should be asked of all managers is: What are the things you 
do, every day, In managing your workers, to prevent their making mistakes? The 
response will indicate the current defect prevention system in the unit. 

The process implemented by the GSL Branch, DPO for the manual interest 
billing and reinsurance activity should serve as a model for this type of corrective 
action. In both cases, the Branch Chief took responsibility for identifying and 
correcting basic problems. Each of the aitical activities could be managed 
similarly by Branch Chiefs and other managers on a project basis. 



^Ibid. 
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Collate «nd Analyze Findiiigs 

A significant amount of inf onnatlon will be generated from the prior steps. A 
critical task is to put these results together into a meaningful report* The re^rt 
should locus orv 

o Cause and effect relationship; 

o The effects of the current system; 

o The marginal changes that can be made to improve performance; 

o Assessment of the likely effects of possible marginal change. 

Most of the s^iously negative effects of the current student aid delivery 
system can be improv^l throMgh marginal improvements to the current system. The 
critical activities could all be the si^ject of this type of analysis. 

Install Ongoing Corrective Action System 

Once the Quality Council has been through this cycle, the l>asic parameters of 
the ongoing corrective action system can be defined and implemented. In fact, this 
process can become an annual corrective action process that can lead to ongoing 
refinement and improvement of the student aid delivery system. The implementa- 
tion of corrective actions In each of the criticai activities would not lead to 
improved overall performance of the delivery system. 

PRIORITIES FOR ACTION 

This report has summarized a two-year quality system design and technical 
assistance task that has focus«^ on the develofKitent of an ongoing Quality improve- 
ment program for the Office of Student Financial Assistance. In order to assure the 
continued miplementation of the quality improvement program, the following prior- 
ities for continued action have been identified: 

o Assign responsibility for quality program; 
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o Design and develo|)ment oi quality control management information 
tyftcm; 

o Implement corrective action co mp onent of quality improvement 
program; ^ 

0 Continue to develop selected targets of opportunity. 
Responsibility for Quality Improvement Profram 

It has been proposed in this chapter that the Deputy Assistant Secretary for 
Student Financial Assistance designate a senior official responsUkte for the OSFA 
quality improvement program. Alternately, a Quality Coimcil can be appointed to 
oversee the implementat^ of the quality toprovement program. 

Quality Control Management Information System 

As indicated in Chapter 3, the Quality Control Management Information 
System (QCMIS) is the critical implementation lii^ of the measurement component 
of the quaUty improvement program. The QCMIS shared emphasis: 

o Refinement of reporting mechanisms from other OSFA quality control 
systems and subsystems, according to the framework that has already 
been established; 

o Systematic analysis of policy and management issues inherent in the 
results of the ongoing measurement sy5*5m; 

o Possible inclusion of the corrective action component. 

Corrective Action Component 

The corrective action component of the overall quality improvement should 
involve all organizational units in OSFA in a systematic quality improvement 
program. It is possible that data analyzed as part of the Delivery System 
Assessment Task, Part n of Stage II, could be used to analyze and prioritize 
corrective action options for each delivery system activity. A blueprint for this 
type of involvement was discussed earlier. The reporting mechanism developed 
from this process could be irxurporated into the QCMIS. 
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Targets of Opperttnlty 

In addition to continuing development ol the reporting mechanisms across the 
OSFA QC syftem, at least two ol the targets of opportunity already considered-^re 
in need of continued technical assistance. Chapter 3 identifies a list of possible 
targeu. This chapter proposes a strategy for developing a systematic framework 
for developing corrective actions for each target. The goal of OSFA should be to 
establish e corrective action agenda using this framework. 
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APPENDIX A 
QCMI5 OVERVIEW 
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This appendix provi<les a framework for developing the OSFA Quality Control 
Management Information System (QCKftlS). The concept behind the QCMIS 
framework is simple but effective. Based on the Pareto principle, which 
hypothesizes that 20 percent of the cases cause eight percent of the problem6, a 

mi 

strategic approach to QC development in OSFA was developed. The strategic 
approach assumes that the most error-prone areas should be selected for QCMIS 
development, and puts the $0/20 principle into action by targeting corrective 
actions on the 20 percent of the cases with the biggest problems. This strategy 
worked effectively during Stage II of the quality control study. 

When fully developed the QCMIS should provide a defined set of inputs, 
processes and outputs for each component, or module, in the OSFA QC system. This 
appendix focuses exclusively on the outputs of the systan. The QCMIS, as explained 
in Chapter 3, would omsist of routine management reports on QC measurement of 
critical activities in OSFA. In addition to reviewing these report formats, this 
overview provides a framework for the Deputy Assistant Secretary for Student 
Financial Aid, or a Quality Improvement Council, to select new targets for QC 
development. 

The overview is divided into three major sectims. The first section considers 
the report formats for the external quality control system. The second reviews the 
report formats and options for the two management level quality control 
subsystems. Finally, the third section proposes a framework for identifying the 
quality control measurement that can be developed for the OSFA student aid 
delivery system. This section includes examples of report formats and sample 
applications of the framework. An objective of Stage lU of the Pell Grant Quality 
Control Study will be to develop fully the specification for the OSFA QCMIS. The 
purpose of this appendix is to identify report formats for the QCMIS. It does not 
specify the system. The final section does, however, provide a framework that can 
be used to specify the QCMIS for critical delivery system activities. 

EXTERNAL MEASUP^ENT SYSTEM 

The long-range goal of the OSFA Quality Improvement Program is to have 
ongoing external QC systems for all three major program components: Pell, GSL, 




and Campui-Based. Currently only one oi these has been devcJoped— the Pell 
external QC measurement system. Stage I oi the Pell QC study actually provided a 
measureivient oi the error levels for Pell. As part oi Stage n of the Pell QC study, 
guidelines were developed lor wooing Pell QC measurement systems. During Supge 
in of the Pell QC study, this study is being replicated. It is possible that the Pell 
QC study could be extended to include the Campus-Based programs. The GSL 
programs would require a separate QC study since it would probably include lenders 
as well as applicants and postsecondary institutions. The report formats for the 
three components are described below. 

For the Pell Grant Program, the key summary report for the QCMIS would 
give the summary estimation of institution and student error. The format for this 
report, as presented in the executive summary of Stage I of the Pell Grant QC 
study, Figure A-1, includes summary estimates for the net and gross number dollar 
error, the percent of individuals or institutions with errors, and the mean error 
amount. The table also provides an indication of the amount of overaward and 
underaward. The PeU QC studies, of course, provide more detaUed results including 
specialized analyses of policy issues, but these basic summary reports can provide 
the basis for QCMIS reporting. Additional summary tables would provide a basis for 
QCMIS reports which include: 

o Breakdowns for the size of disbirsement errors (see Figure A-2 for an 
illustration); 

o Impact of selected incorrect application items on grant disbursements 
(illustrated in Figure A-3); 

o Summary of institutional error incidence (illustrated in Figure A-'f). 

For Campus-Based programs, the table structure would be similar, although 
the definitions of error, both for applicants and institutions, would vary 
considerably. As part of Stage HI of the PeU Grant QC study. Advanced Technology 
is refining a list of error for institutional and student errors for the Campus-Based 
programs. When developed, these can be further refined into summary report 
formats for the QCMIS. 
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For the CSL program these basic report formats would have to be revised to 
include lenders and guarantee agencies as well as students. The definition of errors 
will also vary for the CSL prop-am, 

MANAGEMENT IJ5¥EL REPORTS 

* The design OSFA Quality ImfM-ovement program design includes two basic QC 
measurement modules that operate at the management level in OSFA. Both have 
elements that cut across programs and subsystems in the delivery systems* These 
are a vender processor quality contrd system and a goals and objectives quality 
control system. An overview of the types of reports that could be developed for the 
QCMIS follows. 

Vend<M'/Proce$s(M' QC 

The vendor /processor quality ccHitrol subsystem is actually ccwn prised of a 
series of contracts fw the processing fimcticm fa^ OSFA programs. The internal 
reporting forms developed for rep«'ting on vendor /processw contracts are contained 
in Figure A-5. They include two types of performance data: 

o Data on a series of key processing measures; 

o Data on exceptions (measures of performance that arc operating outside 
of established tolerance levels). 

The form is designed fw middle-level managers; therefwe, only critical 
indicators of contractor performance are reported so that managers do not have to 
wade through excessive information to f imi the data they are interested in. 

The section on key measures selects a nvspber of key processing activities (in 
Pell, for example, these might be applications processing, correcticms processing, 
correspondence service, and telephone service), and for each one, reports 
information on: 

o Processing volime; 
o Processing costs; 
0 Error rates* 
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In addition, other key measures are prascnted and compared to predetermined 
standards to measure processinf efiiciancx. (In PrJi, other key maasurcs may 
include the number of applications in the system more than 20 days, the number of 
corrections in the system more than 10 days, the percent of applications with mere 
than one transaction, and the percent of edit identified errors that are real errors.) 
For each measure, data are reported for this processing period, the prior processing 
period, and the year-to-date* This use of trend data allows the identification of 
potential processing problems before they become actual problems and the prompt 
initiation of corrective actions. 

The section on exceptions Is divided into the various processing steps and 
s;ibsteps. These steps and substeps are: 

Production control; 

information receipt, 
data entry, 
data edit, 
« compute, 

doaimeiit production and mailing, 
corrections. 

Fiscal control; 

Software quality assurance; 

Productivity control; 

Reporting process; 

Corrective action process. 

Since there are so many potential quality control measures in a 
vendor/processor contract, this section of th^ report is actually a table shell on 
which the monitor Usts only measures that are operating outside of tolerance 
ranges. Depending upor. contractor perfonnanoe, the number of exceptions may run 
from zero to several dozen. This table shell approach provides a much more 
readable and useable format for managers than pre-printing all possible performance 
measures. 
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OSFA Coals aiNl Ob)«ctiv«s System 

The basic reports for the current OSFA goals and objectives system consists of 
Gantt charts lor all activities and tasks required to delive student aid. As a p£t.of 
the Stage 0 Pell Grant QC itudy, Advanced Technoioo* did a QC enhancement study 
of this system. This enhancement study focused on testify the network concept for 
the' delivery of student aid program'*. It was concluded that the network approach 
could generate a variety of strtmary management reports. These include: 

o A cost control Gantt chart for each phase of the process or function that 
displays the scheikile status of activities to be accomplished within the 
Baseline Cost Estimate CBCE) funds available; 

o A cost status report that lists fixed and variafaae costs with associated 
aciivities; 

o A resource analysis report which describes, in dollars, the status of funds 
(budgeted or actual) within each cost center of the BCE; 

o A cost-versus-time plot of budget-versus-obiigated funds for each cost 
center of the BCEj 

o A critical path of the activities network that routinely reports planned 
and actual achievement Of project activities. 

The OSFA Coals and Objectives system has the capacity to meet OSFA 
decision support needs, cspedaUy if the network approach is used. OSFA's decision 
support needs are iUustrated in Figure A-fi. All of these needs can be met by an 
automated networking system, such as the PRIME/VISION system used for the QC 
enhancement study* 

At a poUcy-ievcl, the Goals and Objectives system using the network approach 
can provide summary reports on the status of the delivery system, status reports on 
programs and sid>systems, simulations of decision plans, and resource tracking and 
allocation reports. These reports can be tailored to meet the routine reporting 
needs of senior administrators or to provide early warning of systen breakdown. 
Such decision support systems can also meet the detailed reporting needs of miOdic 
managers and operatims managers. 
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DEUVERY LEVEL QC MIS FRAklEWORK 

Durii^ Stage H of the Piell Grant Quality Control Study, OSFA experim^ted 
with the strategic approadi to QC lystem development* The methodology proved 
efiective at improving the capacity of operating units in OSFA to measure error and 
to design c<»'rective actions. The major limitations of this a{^roach were; 

o OSFA had limited capability to identify error-prone points in the 
delivery svstem, since there was not a comprehensive QC framework; 

o Onve an area was selected for QC developmcnti the actual strategies for 
developing the QC measurement system had to be identified. 

Based on t^(t f xperience during Stage it Is now possible to suggest the basic 
c^^-nents of such an analysis system. Two parts of the Stage U study can be used to 
:v j^cst such a framework. First, the framework developed in the Delivery System 
Assessment Task, Part 11 of Stage U, provides a reference point and methodology for 
developing a compr e hensive QCMIS framework, one that can be used to ic2entif y QC 
measures for key activities. Second, the experience with targets of opportunity 
during the Part I study can be used to help identify strategies for developing these 
QC measures. 

A framework for developing a comprehensive QCMIS for the OSFA delivery 
system is proposed in this section. First, however, it is necessary to review the 
framework used in the Delivery System Assessment Task. 

Delivery System Assessment Fr a mework 

The Delivery System Assessment Task used a very detailed methodology to 
specify the current student aid delivery system, to develop a model to measure iu 
effects, and to apply this model to the current system. This same methodology can 
be adapted to develop a QC measurement framework. 

The methodology for developing the analytic model for the Delivery System 
Task required faking a series of analytic steps. These were: 
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o Step It Sfwciiy tht onrcnt dtUvcry system in the iorm oi input- 
process-output OPO) chains. 

o Step 2s Develop Independently a detailed list of program features for 
eadh program* 

o Step 3t Oetm)lne which program features influence each deliv«'y 
system activity. 

o Step 4s Determine the Intervening variables that are relevant to each 
delivery system activity. 

o Step 3t Detmnine wfiidh effects are influenced by tadh delivery system 
activity. 

o Step 6s Develop measures for each effect at each delivery system 
activity. 

o Step 7s Identify existing data sources or develop new data sources for 
each measure. 

o Step Ss Identify methods of analysis for each effect at each system step. 



This process resulted in a preliminary specification of the entire student aid 
delivery system. This specification Is currently being revised. The refined list of 
delivery system activities is contained in Figure 3-^ in Chapter 3. A series of 
reports wei t generated during the course of this analysis. These were: 



o A review of previous approaches to delivery system Issues, which 
resulted in a context papa-} 

o A preliminary modely which provided the logic for developing the 
detailed model; 

o A specification of the current qmm, Including program features (from 
laws, regulations, and administrative decisions) and system steps (in the 
form of input, process, output chains) for each activity in the delivery 
system; ^ 

o A general assessment model, which identifies Intervening variables and 
effects for each delivery system activity; 

o An analytic agendat which Identifies measures, data sources, and 
methods of analysis for each effect. 



The three key reports in this series, as far as the methodology for the QC 
measurement framework is concerned, arc the specifications, the general model, 
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and the analytic agenda. The relationship between the three documents i$ 
Ulurtrated in Figure A-7. The tpedlication document identified the program 
leatures and dtoliyery system fteps associated with each activity in the current 
delivery system. Fifure A-4 Uiistrates the iormatkm lor the spediication of^^ 
delivery system activity. The Cenerai Assessment i^lwdel identified the intervening 
variables and effects for each activity. Figure A-9 provider an illustration ol the 

0 

format for this analysis for the same activity. The analytic agenda further extended 
this framework by identif yli^^ measures, data sources, and analysis methods for each 
effect. This is illustrated in Figure A-10. The methodology proposed below applied 
this same methodology. 

The General Assessment Model is being applied currently in the analysis of the 
effects ol the current delivery system alternatives. The steps required to apply the 
model are; 

o Evaluate the current system, including an cstkiaticm of baseline effects 
on all partic^>ants{ 

o Identify delivery system alternatives, based on review of past proposals 
and community ii^iuti 

o Specify selected altematlves, including program features and system 
steps; 

o Assess altcsmatives, including estimation of differential effects on 
participants; 

o Spediy intent, p«'h^ in different ways; 

o Rank alternatives according to specification of intent. 

If the QC development framework is developed^ then it can be systematically 
applied, using the strategic approach, to high error-prone points in the delivery 
system. In fact, the results of the assessment of the current system can be used to 
identify error-prone points. 

Delivery Level QC Aleasurement Frameworlc 

The framework and arialysis completed at part of the Delivery System 
Assessment Task provide the basis for the proposed framework for the delivery level 
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ILLUSTRATION OF FORHAT FOR SPECIFICATIOH OF H 
THE CURRENT DELIVERY SYSTEM 
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BEST COPY 



PELL GRANT COMPONENT 

1. PRE-APPUCATION SimSYSTEM 
1.1 IMBet Fbrecastbis Actlvhr 



INTERVENtNG VARIABLES 



• Funds iivallable for activity 

• Timellnesf and content of relevant political decbkms 

• Degree of change from previouf year 

• Technolo gy avallable/tised 

• Staff productivity 

• Data base available/used 



EFFECTS 



a. Admbilstratlve Costs (Federal Government) 



t 



b. • Accuracy of forecasting activities 

o Integration of forecasts with other activities 



• Pdllcy decisions of Aihntnlstratlon, Congress 

• Forecasting technology available/used 
m Similarities across programs 



b. Fund Control (Federal Government) 



c. Integrallon Across Programs (Federal Government) 



• PMIctablllty of program changes 
a Avallabllltyt accuracy of data used 

• Predictability of changes In the participant 
population 

• Forecasting technology available/used 



d. Fund Forecasting (Federal Government) 



ERIC 



e. • Type, avallabllltyy completeness, tlnfellness, 
accuracy of data used 
• Forecasting technology available/used 



f, • Accuracy of forecasting acthfitles 

• Policy decisions of Admbibtratlon/Congress 
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Availability of Program Information 
(Federal Government) 



f 



f. IMstrlbutlon of Aid 
(Applicant/Family) 

FIGURE A-9 



116 



DRATT 



e; 



:3T COP/ 



EFFECTS 



i a. Admlntstratlve Costs 
(Federal Government) 



b*! Fund Control 

' (Federal GovemmenO 



9» 

I 



Fund Forecasting 
(Federal Government) 



PELL GRANT COMPONENT 

I* PRE-APPUCATION SUnSYSTEM 
1.1 Budget Forecasting ActNltr 



MEASURES 



Costs to Federal government 
of developbig buc^t lore- 
cast 



• Supplemental appropriations 
needed during the year or 
the tumhack of funds 
occurring «t year^ end 



• Difference between bw^et 
forecast and actual program 
expenditures for the year 



DATA SOURCES 

• DPPD and OPBE budgets 

• Interviews with appro- 
priate ED pe r sonn e l 



• Budget data 



• PIMSdata 

a Budget data 

• AppTlcant-liased model 
t ISFAM 



ANALYSIS METHODS 



• From bitervlews and budget documentSv 
detcrndne Federal admbdstratlve 
costs attrttiutable to the budget fore- 
casting function 

• ttetertnlne magnitude and fre<|uency 
of both supplemental appropriations 
requb-ed and funds returned for past 
years* using budget data 



Calcula^ difference between budget 
forecast prior to year*l end and actual 
expenditures for that year 
Analyse the bnpact of chaises in the 
delivery system on accuracy of budget 
forecasts by sbmitatbig prior year 
data, ushig the applicant-based model 
or ISFAM 



FIGURE I 

ILLUSTRATION OF THE FORHAT FOR 
THE AHALYTIC AGENDA 
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QC measurement Iramework. There are two key ftcps in the dcveiopment of the 
framework. 



First, the spccifiCBtions document provides a starting point for speci^ 'ingJthe 
types of quaihy control that can be developed ior each critical activity. Figure 
A- 11 illustrates the iramework lor doing this. The specification of the current 
system identifies the program features for each activity, as well as the delivery 
systems required to complete each activity, in the form of inputs, processes, and 
outputs. It is possible to use those spediications to develop the QC framework. 
Specifically, for each critical activity, it would be necessary tot 

o Identify the types of quality control that apply, including receipt control, 
productivity control, and fund controli 

o Identify the specific QC measures that apply such as timing of receipt of 
documents and completion of reviews for production control; 

o Identify potential data sources for all measures; 

o Identify the specific strategics that could be used to develop the QC 
measurement component. 

For illustrative purposes these steps have to be applied to two delivery system 
activities. Figure A-12 applies it to activity 2| student application examples 
illustrates how the vendor/processor function for Pell would be iilustated in this 
framework. The second illustration is Figure A*13, activity 6*6, CSL claims and 
collections, which is an activity thWt Is currently being reviewed for QC. 

The second key element of th^ framework will be the reanalysis of the results 
of the assessment of the currert delivery system, which has been recently 
completed. The assessment of the (current sjrstem provides baseline measures for 
e«h deU».ry v*ten. ellect »d J^yz« how indivldu.1 «tiv.tle. ocmrlbut. to 
each effect. A reanalysis of this dafi could help inform the QC framework in two 
ways. These are: 



o The tfialysis would be used to identify the measurement strate5ie.s for 
each activity; 

o It could also be used to identify error-prone points in the delivery 
system. l 
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DRAFT 



T 



SPECIFICATION OF THE CURRENT 
DELIVERY SYSTEM* 




DELIVERY SYSTEM LEVEL QCMIS 
MEASUREMENT FRAMEWORK^ 











L 



Features/ 
Activity 



^ Laws 

• Regutetlom 



• Administrative 
Decisions 



1 



Delivery System 
Ste|is 



• inputs 



• Processes 



• Outputs 



r 



1 



Type of QC 




QC Measures 




Data Sources 




Strategy 


• Receipt 
Control 

• Productivity 
Control 




• Timing 

• Performance 




a PTocedT*^ 
• Behavior 




Method for 
dev«loping QC 
MettMirement 
System 








• Fund 
Control 




• DoiiM'S 




• System 
Performatioe 







* A preliminary specification for the current student aid delivery system was developed bjr Advanced Technotogy* Inc*, as part of the 
Delivery System Assessment Task. This specification is currently being revised to Incorporate Input from 05FA personnel. 

^The format for this report will be similar to the "Analytic Agenda to the Current System," developed as p9rt of the Delivery System 
Assessment Task. The content will reflect the findings of that analysis. 



FIGURE A-l I 
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CONCEPTUAL FRAMEWORK FOR THE 
DELIVERY SYSTEM LEVEL QCMIS 
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TYPE OF QC 
Control 



MEASURES 



• Onrnt of i^ipHcatlons received 



• Backk)g of applicattons 



• Processing time from receipt to 
data entry 



• Count of applications flagged by 
cursory edit 

• Count of the number of times a 
particular edit Is used 

• Percent of errors In Kwtlng 
applications by type 



DATA SOURCES 
• QC and QA sample 



• Contractor data base 
and QA sample 

a QC and QA sample 



a Contractor data base 



• Cantr»:tor data base 



• QC and QA sample 



OR APT 
STRATEGY 



Comp^e counts of In- 
coming documents to 
counts at various pro- 
duction steps 

Analyze efficiency of 
processing by measuring 
badclogs 

Take Independent sample 
of documents received, 
assign ID number* record 
date and time of entry 
into system* and track 
through processing stream 

Analyiee accuracy of edit 
procedures 

Analyze reliability of edit 
procedures 

Analyze the accwacy of 
Information sorts 



nnoductivity 
Control 
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Count of i^llcations entered 
ver^ counts at various 
automated processing steps 

Count and percentage of errors 
by keystroke* data element* and 
form 



• Contractor data base 



• QC and QA sample 



Analyze whether applica- 
tions arc being lost 
after entry 

Take sn independent 
san^le of input docu-* 
ments wkI compare trans> 
formed data to original 
documents for accwacy 



FIGURE A-12 

EXAMPLE f 1 OF DELIVERY LEVEL <QCMIS) FRAMEWORKS STUDENT 
APPLICATION ACTIVfTY (FOR AN APPLICATION PROCESSING CONTRACT) 



I 
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TYPE OF QC 

Prodoctlirlty 
Control (Cont.) 
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MEASURES 



• Backlog o! applkatkms 



• Processing time Irom receipt at 
keypufxrh to completion of data 
entry 

a Count of telephone inquiries 



• Percent Incoming aills placed on 
hold 



• Average length of time calls 
placed on hold 



• Count of number of calls per 
application 



DATA SOURCES 



• Contractor data base 



• QC mkI QA sample 



• Contractor data base 

• QC and QA sample 



• QC and QA sample 



• Contractor data base 



• Accuracy of answerlf^ applicant • QC and QA sample 

questions 



DRAFT 
STRATEGY 



Analyze efficiency of 
data entry by measuring 
backlogs 

Take Independent sample 
and monitor processing 
time from receipt at key- 
punch to data entry 

Analyse telephone inquiry 
logs 

independently monitor 
operators and record 
number of calls placed 
on hold 

Mependently monitor 
operators and record 
length of time call Is 
on hold 

Analyze telephone Inquiry 
logs and assess how well 
overall processing system 
works and how under- 
standable procedures are 
to fl|)pllcant 

Independently monitor 
operators for accuracy 
by phoning In typical 
questkms and recording 
accuracy of response 



FIGURE A-12 (ContbMied) 
EXAMPLE f I OF DELIVERY LEVEL (QCMI5) FRAMEWORK* STUDENT 
APPLICATION ACTIVITY <FOR AN APPLK:ATX>f<l PROCESSING CONTRACT) 
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TYPE OF QC 

IVodoctiTity 
Control (Cont.) 



Fund Control 



MEASURES 



• Count of letters received 



• Count of responses to letters 



0 Average number of days before a 
response is mailed 



9 Accuracy of answering applicant 
(|uestions 



Percent of output data con^ted 
accurately 



Actual versus budgeted processing 
costs by period of performance 



DATA SOURCES 

* 

• Contractor data base 



• Contractor data base 



• QC and QA sample 



• QC and QA sample 



QC and QA sample 



• Contractor data base 



DRAFT % 
STRATEGY 

Analyze receipt control 
logs 

Analyse accuracy of 
correspondence service 
by assenting percentage 
of letf ^•rs receiving a 
timely response 

T«ke independent sample 
of incomli^ letterSf 
assign in number, record 
entry date and time, and 
monitor until response Is 
mailed 

Write sample letters and 
monitor accwacy and 
timelln<»M of responses 



Take indtrpcndcnt sample 
of applications and 
manually replicate com- 
putation of nutput. 
Compare with automated 
con^te steps. Analyze 
accuracy for various 
voltmies of applications* 

Analyze cost trends since 
volume vfl«-i€S over course 
of year affecting cost. 
(For CPFP contract only). 



o 
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FICVKE A-12 (Contfamed) 
EXAMPLE f 1 OF DELIVERY LEVEL (QCMIS) FRAMEWORKS STUDENT 
APPLICATION ACTIVITY (FOR AN APPLICATION PROCESSINC CONTRACT) 
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T YPE OF QC 

Fund Control 
(ConU 



MEASURED 



DATA SOURCES 



• Actual verfMis budgeted firocessing • Contractor daU base 
costs by cost center 



• Actual unit costs versus bu(^eted # Contractor data base 
unit costs 



DR A F T 
STRATEGY 



Analyze cost trends by 
cost center. (For CPFF 
contrjMTt only) 

Analyr.e unit cost by time 
period since cost may 
vary based on volume 



I 



FIGURE A>I2 (Contbwed) 
EXAMPLE #1 OF DELIVERY LEVEL (QCMIS) FRAWEWORKr STUDENT 
APPLICATION ACTIVITY (FOR AN APPLICATION PROCESSING CONTRACT) 
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TYPE OF QC 
Meceipt Control 



it* 
o 



MEASURES 



• Count of claims received 



• Count of collecttons received 



• Processing backlog of claims 



• Processing l>acklog of collections 



Processing time from receipt to 
data entry (claims) 



• Processing time from receipt to 
data entry (collections) 



• Count of claims flagged by edits 



DATA SOURCES 

• Processing informatkwi 
at Student tomt Pro- 
cessing Center (SLPC) 



• Processing Informatloii 
at DPO 



• Processing information 
at SLPC 



a Processlf^ Information 
at DPO 



• Processing Information 
at StPC and QC data 



• Processing Information 
at DPO and SLPC plus 
QC data 



• Processing Information 
at SLPC 



DRAFT 
STRATEGY 



Compare counts of iiS9 
forms received to counts 
at various production 
steps 

Compare counts of 1 189-2 
forms rer^lred to counts 
at various production 
steps 

Analyze efficiency of 
IW processing by 
measuring 1189 form 
backlog* 

Analyse efficiency of 
1189-2 processing by 
measuring 1 1 89-2 form 
backlogs 

Assign all claims an 10 
numbeTf record date and 
and time of entry into 
system^ and track throtigli 
processing stream 

Assign all collecttons an 
ID number, record date 
and time of entry Into 
system, and track th nigh 
processing stream 

Analyze accuracy of edit 
procedures for 1 189 forms 
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FIGURE A-U 

EXAMPLE #2 OF l>ELIVERY LEVEL (QCMiSi FRAMEWORKi 
CLAIMS AND COLLECTIONS ACTIVITY 



I 
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DRAFT 



TYPE OP QC 

Receipt Control 
(Cont.) 



MEASURES 



DATA SOURCES 



• Count of collections flagged by edits • Processing Information 

at OPO and SLPC 



STRATEGY 



Analyze aoruracy of edit 
procedures for 1 189-2 
forms 



I 



IVoductlvlty 
Control 



• Count and percentage of errors 
by keystroke^ data element, and 
form 



• Backkyg of claims and collectkMis 



Count of claims and collections 
received compared with counts of 
claims paid and checks desposited 



• Accuracy of rebalancing I IS9 totals 
to reflect rejected transactions 



• Accuracy of resolving adjustments 
to balances 



• Processing informatkm 
at SLPC and QC data 



• Processing Information 
at SLPC 



• Processbig biformatlon 
at SLPC and DPO 



t Processing information 
at OPO and QC data 



• Processif^ information 
at OPO and QC data 



Take Independent sample 
of claims and collections 
and compare transformed 
data to original 1189 and 
1189-2 forms 

Analyze efficiency of 
1189 »id 1189-2 data 
entry by measuring 
backlogs 

Compare counts from 
control logs at SLIK: and 
DPO to claims and 
coilectksm data at Office 
of Financiaf M a nagement 
Services (OFMS) 

Design formal staff pro- 
cedures and provide 
training on procedures* 
On sample basis, validate 
computations 

Design formal staff pro- 
cedures and provide 
training on procedures. 
On sample basis, validate 
computations 
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FIGURE A-13 (Cont.) 
CLAIMS AND COLLECTIONS ACTIVITY 
EXAMPLE 02 OF OELIYERY LEVEL (QCMIS) FRAMEWORK 



I' 



133 



DRAFT 



TYPE OF QC 

IVoductivity 
Control (Cont.) 



MEASURES 



DATA SOURCES 



STRATEGY 



• Accuracy and timeliness of responding • Processing information 
to questions from state agencies at DPO and QC data 



• Accuracy of preparli^ voucher 
(1166 form) 



• tYocessing informatton 
at OPO and QC data 



Talce san^le of corres- 
pondence received from 
state agencies and moni- 
tor accuracy of response 
and response time 

Design formal staff pro- 
cedures and provide 
training on procedures* 
On sample t>asiSt validate 
Computations 



Nimiber of overpayments and 
underpayments 



• Mumber of duplicate payments 



• Accuracy of collections data 



• Processing information 
at DPO and QC data 



• Processing information 
at BCS 



• Processing information 
at DPO and QC data 



Design formal staff pro- 
cedures and provide 
training on procedures. 
On san^ basis, verify 
claims paid against 1 1S9 
form 

Analyze accuracy of edit 
routine to prevent dupli- 
cate payments 

Design formal staff pro- 
cedures and provide 
training. On sample 
basis, verify collections 
received against 118^2 
form. Design procedures 
to separata collections 
into principal and Interest 
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mCURE A-13 (Cont.) f* 
CLAMS AND COttECTIONS ACTIVITY 
EXAMPLE 02 OF DELIVERY LEVEL (QCMIS) FRAMEWORK 
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TYPE OF QC 

Fund Control 
(Cont4 



MEASURES 



DATA SOURCES 



• Timeliness of claims and collections • QC data 



Percentage of vouchers reprcientlng 
valid obligations 



Processing information 
at DPO and OFMS 



D R A r T 
STRATEGY 



Develop method for 
accurately aging accounts 
by state 

Unifjr data bases of DPO 
and OFMS 
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FIGURE A-13(Cont.) 
CLAIMS AND COLLECTIONS ACTIVITY 
EXAMPLE #2 OF DELIVERY LEVEL (QCMIS) FRAMEWORK 
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Both oi these features are critical to the QC measurement framework. Ont of 
the major obstacics impeding the widespread application of the strategic approach is 
that the specific factors that should be measured— the types of QC that apply, the 
possible measures, and the strategies ior de^loping the measures— musU>e 
developed on an ad hoc basis. This framework, building on the experience gained in 
the earlier analysis would Identify these factors*^ This identification of 
measurement strategies could be Informed by the analysis that has already taken 
place. 

Additionally, the reanalysls of the evaluation of the current system would 
identify error-prone points in thm delivery system* This analysis could focus on 
identifying the activities in the delivery system that cause the most problems to the 
Federal government, states, institutions, and applicants. 

In summary, the near completion of the Delivery System Assessment Task 
provides OSFA with an excellent opportunity to develop a solid and comprehensive 
QC framework for the student aid delivery system. The results of this analysis 
provide a starting point, methodology, and data base for such an exercise. 
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APPENDIX C 
CSL MANUAL INTEREST MLUNC FORMS 
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TABLE OF CONTENTS 



CSL: MANUAL INTEREST BILLING FORMS 



Ch«ckUst (to ascertain errors in each individual transaction) 
TaiJy sheet (add up errors found In each individual transaction) 
Summary Sheet lor individual technidam 
SuTT.mary Sheet for OSFA-QCMIS 
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DRAFT 



Pagel 

Control Number Todays date:,^ 

J4D San^te period: 

Technician Fromi / 

Date of Receipt Month Year 
Total $ Payment To: / 



Montn Ye^i;^ 

CSL BRANCHAXK) QUAUnf CONTROL CHECKLIST FOR 
MANUAL PTTEREST PA YMENT5 
OCSAiiPLING 

fcwtructiongt As each step is completed,' initial appropriate column. U step is not applicable, 
write "N/A" in initial column. Place check mark in error column if item is incomplete or 
inaccurate. If error is measurable k» dollars, write dollar ammsit of error in appropriate 
column, usii% a plus (•*•} for errors above the correct amount and a minus (-) for errors under the 
correct amount. 



rTEM INITIALS ERROR 
1. RECEIPT CONTROL 

1.1 Review 799 Control Sheet for completeness: 

a. Date of receipt (at SMA in Norfolk) 

b. Penalty date (30 days after date of receipt) 

c. Lender identification number 

d. Quarter ending date 

e. Reason for rejection 

f. Control number 

g. TBS date of receipt 

h. aerk«s initials 

1. Person assigned ^ 

)• Date assigned 

1.2 Review case entry on Control Log for completeness 

a. Control number (sequentially assigned) ] 

b. Current date (TBS date of receipt) 

c. Lender identification number 

d. Person assigned (based on type of payment) 

e. Penalty date 

1.3 Is information on Control Sheet and Control Log 

different? (If yes, check "error" column) 
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Control Number 
LID 

TschiUdan 

Date of Receipt 
Total $ Payment 



DRAFT 
Page 2 



CHECJOST TOR lAAWAL INTEREST PAYMENTS 



ITEM 

1.4 Is information on the Control Sheet and the 799 form 
itself different? (If yes, chedc 'terror* cniumn) 

1.5 Is current date on Control Log more than five working 
days later than SMA date of receipt? Uf yes, check 
••error* coluim) 



INITIALS ERROR 



Z 799 COMPLETENESS 

2.1 Review the 799 form for co npieteness 
Identification Section 
a. Lender name 
h« Lender address 

c. Employer identification numbe* 

d. Lender telephone mimber 

e. Billing period 

f. Billing period ending year 

g. Method 
Section I - Interest 

h. Number of loans 
L Principal balance 

j. Average principal balance for period (not 
required if actual accrual used) 

k. Interest due 

L Subtotal interest due 

m. Adjustments (if applicable) 

n. Total interest due 

Section II - Special Allowance 

o. Year 

p. Average principal balance 
q. Actual principal balance 
r. Method 
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DRAFT 

, Control Number P*Se 3 

• Technician ' 

0* Receipt 
J' Totai % Paymvnx 

CHECKLIST FOR MANUAL INTCREST PAYMENTS ^ 

INITIALS ERROR 

If adjustnient claimed, 

s. Average principal balance reported 

t. Corrected average principal balance 

u. Quarter ending date ^...^ . 

Section in - Origination Pee 

V. Principal amount — 
w. Adjustments (if applicable) ...^ 

X. Difference 

y. Origination fee n ■ 

2.2 If tax exempt lender, are supplemental forms attached? 

(If no, chedc error coiumn) 

2.3 If prior period interest adjustment claimed, is explan- 
ation included or documentation attached? (If no, cM< 

error column) ■■. 

2.U If prior period special allowance adjustment claimed, is 
explanation included or documentation attached? (If no, 

check error column) 

2.5 If prior period principle balance adjustment claimed, is 
explanation included or documentation attached? (If no, 

check error coluim) 

2.6 Are tapes from original calculations of the interest due, 
the special allowance, and the total payment attached? 

(If no, check error column) 

X 799 ACCURACY 

(In this section, the error column ihouid be checked only 
if the original amount is more than $1.00 above or betew 
the correct amount. Recalculations should be done more 
than once, especially if inconsistencies exist. Recal- 
culations should also be checked against adding machine 
tapes). 



/ 



ERIC 



^mtrol Number 

Date of Receipt 
Total $ ?By?n«rm 
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CHECKUST FOR MANUAL 1NTER3EST PAYMENTS 



INITIALS ERRC» $ AMOUNT 



3-1 Origination Fee Check 

Is Section m, line 4, 5% of line 3? (If 
notf cbecic error Golumn and oiter amomt 
of error) 



3«2 Interest Check 

a. If method is actual accrual skip to 
item 3.3. 

b. If method is average quarterly balance* 
multiply average amount (column 7) x 
interest rate x 90/360 

If method is average dally balance, 
n^ultipjy average amount (column 7) x 
Iniferest rate x (number of days in ' 
quarter)/363. 

3.3 Recalculate total interest claimed, adding 
or subtracting partial si^>sidies or prior 
year adjustments were applicable 

3.4 Special allowance check: recalculate by 
multiplying average amoimt x special allow- 
ance factor 



Total $ vahie of errors lor Sectksn 3s 



NON^UPUCATION OF PAYMENTS 

4.1 Run (or obtain copy of) current lender search 
and verify that no payments for period were 
processed to lender via automated system (If 
duplicate payments exist, check error column 
and tnter amount of ck^licate paymmt) 



(f) 
(-) 
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Control Number 
UD 

Technician 
Date ai Recent 
Total $ PayTT»«nt 
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rHEOCLIST 1^ MANUAa, B4TEREST PAYMENTS 



ITEM 



♦.3 



Examine file of manual payments for lender 
and verify that no other payments for 
period were processed to lender via manual 
svstenu (If di^Ucate payments exist, 
dieck ^or column and enter amount of 
duplicate payment). 

U duplicate payments are found, check 
cancellations and retimed checks to see 
if corrected. If duplication was corrected, 
place check mark here: 



INITIALS ERROR S AMOUNT 



Total $ value of di^Ucate payments: 
(Include payments that were later corrtxrted) 



5. TREASURY PAYMENT VERmCATlON 

Examine copy of SF1166 in file. Verify that 
amount paid agrees with 799. Review to 
ascertain that all inf ormatiai was recorded 
correctly (name, etc). Check error column 
if any items on the 799 do not match the 
SF1166. If amount paid does not agree, 
enter dollar amount of difference. 



6. PROCESSING TTMEUNESS 

Compare certification date on Control 
Sheet to penalty date on Control Log. If 
certification date is later than p«ialty 
date, check error column, and enter amount 
of penalty payment from documentation 
attached to 799. 
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Sample Period: 
Fromj 



Month 



To: 



Month 



Year 



ERROR TALLY 



ET 




NUMBER OF ERRORS 



ITEM NUMBER 



HASH MARKS 



TOTAL 




\ 



Total 
Total 



of tfrors In Section Is 

of cases whh errors In Section 1: 
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ITEM NUMBER 



2A^ 



NUMBER OF ERRORS 



HASH MARKS 



DRAFT 



Page 2 



TOTAL 



A. 



2^ 



Total msnbcr of erran 



in Section^ 
with erron In Section 2i 



ERIC 



07 
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Page 3 

NUMBER CMFERRORS DOLLAR VALUE OF ERRORS 
TTEM NUMBER HAjW MARK5 TOTAL U) {±_ 



3.1 



3.2 b 



3.2 c 



3.3 



TOTAL 



TOTAL 



TOTAL 



TOTAL 



DRAFT 
Page k 

NUMBER OP ERRORS DOLLAR VALUE OF ERRORS 



ITEM NUMBER HASH IIARiCS TOTAL U) {± 

3A 



TOTAL 

3.5 



TOTAL 



Total Etumbcr of errors in Section 3i 

Total number of caaes with errors in Section 3: 

Total dollar value of errors in Section 3s M 
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NUMBER Cy ERRC»S DOLLAR VALUE OF ERRORS 
ITEM NUMBER HASH MARKS TOTAL M (■) 

U.i 



TOTAL 



TOTAL 



4.3 What is the dollar value of 
duplicate payments corrected? 



TOTAL 



*TotaI number of errors in Section 4s 

*Total number of cases with errors in Section 4s 

«Total dollar value of errors in Section 4s 

♦ Include duplicate payments which were later corrected. 



DRAFT 
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NUMBER Of masts DOLLAR VALUE OF ERRORS 

TTEMWUMBER HASH MARKS T^AL (*) 

5. 



TOTAL 



6. 



TOTAL 



Total number of sn^le cases with any error: 



ERIC 



0-11 



1S6 



06FIV 




HOIXSIS OF WtfMRL PflKWsyi' 
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N^ne of TBchnician; 
Name of Supervisor! 



period 1 



From: 



Ho. Yr. 

13o7 YrT 



Period 2 



From: 



Mo. WT 



period 3 



Front 



Frcxns 
1t>s 



Pe riod 4 ^Weraqe 

_ Ftcws - , 

1^ Yr. fio^ *r 



Ho: W7 



itoTW 



1. I of cases included in sanple p6r 
tedmician 

I 2. t of cases In error (nurber of cases 

' in error divided by line 1) 

9 3. Tt>tal dollar value of cases in sanple 

^ per technician 

4. Dollar value of errors 



5. Ptercent of dollar value in errxx (line 4 
divided by line 3) 



(-)$. 
(-1 



% 



(+)$_ 
{-)$_ 



% 
% 



1. 
% 



*Siin of all values on line divided by 4 

personnel actions «*iouM be based only on 4 period averages using a 95» oonfldenoe Interval. 



.1.57 
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OSFA QOgS 
GSL SUBSYSTEM 



siMARy OF srm for I / 

Month Year to Manth Year 



DRAFT 



Nisii»r of Cases Sanpled 



Receipt Omr- tecuracy Duplication Treasury 
Obntrpl pleteness . Verlf . 



6 



Overall 



I 



i of Errocs 



I of Gases idth EbctarB 



Qncor Rate (I of cases with erxor 

divided by total nmter cases 

sanpled) 

Bnxn: Rate Prior Period 

$ Value of Errors N/A 



$ Value of Errors Prior Period N/A 



N/A 



N/A 



•Pleaae notei Error rates t^cen frcsi sables are estijnates only. Oonfldenoe levels and intervals ntist be utilized 
when interpretistg these results. 
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